Introduction

In this article we will show how to run a machine learning task on the cloud. More specifically, we will be showing how to solve a classification problem using Google Cloud Machine Learning solution. The core tools used to build the solution are Python language and TensorFlow open source library. Using Google Cloud Machine Learning services allows you to realize the following tasks:

• ML Engine Job service: Train and evaluate your model using advanced hardware configuration (GPU, multiple GPU, TPU).
• ML Engine Model service: Host and serve your model on the cloud. By having your model served on the cloud, your model becomes scalable and would benefit form dynamical resources allocation that the cloud platform would provide for optimum performance.

Storage at Google cloud is known as a "Bucket". Usually, buckets are referenced using "gs://" initials followed by the project name (i.e gs://project_1/my_bucket). As we will see later, replacing normal file access by "Bucket" file access is one of the possible modifications you have to apply to your code to make it cloud compatible.

Compatibility Check List

• Parameters: In General, any python package or routine can be run on the cloud without the need of specific parameters. Still, as we will show later, there is one special parameter, with the name '--job-dir', that is passed to your Python main routine.
• Versioning: When the Python routine is submitted as a Job, or a trained model is submitted as a Model, you have most probably to tell the Cloud what are the versions of TensorFlow, and Python you want to use. To set the TensorFlow version we have to use the option --runtime-version when using the gcloud utility. To set the Python version, we need to create a configuration file config.yaml that indicate the Python version. An example of such file would as in the following:
  

  trainingInput:
    pythonVersion: "3.5"
  

• Packaging: When your Python task is run as a Job on the cloud the first step that the cloud will perform is installing the Python packages used by your code and mentioned in the import clauses of the source code. Unfortunately, not all packages are compatible with ML Engine virtual machines. Sometimes, it is necessary to load the package using special directives. For example, to be able to use matplot package you have to use the import syntax:
  import matplotlib
  matplotlib.use('AGG')

  Another constraints is that the current directory is undefined when you are running on ML Engine. For example, if your package name is 'my_package' with the files: train.py, eval.py then you cannot import files by using file name directly. Instead, it is necessary to reference the imported file by using explicit package name:
  Wrong: import eval
  Correct: import my_package.eval OR import .eval
  

Model Definition

In our example we will show how to train a model for predicting income level, weather greater or less than 50k per year, according to different criteria. Input data, including the person's income, are rows of the following features:

• 'age'
• 'work class'
• 'education'
• 'education period'
• 'marital status'
• 'occupation'
• 'relationship'
• 'race'
• 'gender'
• 'capital gain'
• 'capital loss'
• 'work hours per week'
• 'native country'
• 'income level'

The proposed solution is to use the TensorFlow Estimator DNNLinearCombinedClassifier. This estimator combines a neural network with a linear regression to produce the predicted income level.

Training on the cloud Example

This example uses the code available at Cloud Samples. After downloading the code, you need to make sure that your local cloud utilities are installed and configured to use your Google Cloud Account and your Current Project.

Next, go to the sample directory

Download the Data

Create the bucket on the cloud, your credentials should have been already saved on local machine

Copy the data

Submit the job using python 3.5 -specified in the Config file config.yaml- and Tensorflow version 1.4. The --scale-tier STANDARD_1 option specifies to create a training job that will use 1CPU + 1GPU

Stream training output to your local machine:

At the end of the training you should be able to read the evaluation accuracy that is equals to:

Once the job has terminated successfully, you can check training output

The output should be similar to the following:

Tensorflow checkpoint(s) : these files represents the standard Tensorflow way of saving training states and results. These files can be used to visualize the model graph using TensorBaord, and/or restart training at later stage. It is a good practice to save these files for later examination. Still, Checkpoint file format is is not compatible with ML Engine Model service. So you do not need these files to host your model on the cloud

The export directory contains the actual, complete and final model data that can be used by ML Engine model service. This is SavedModel file format that contains all the necessary information to serve your model on the cloud. The main difference between checkpoint file format and SavedModel format is that SavedModel contains MetaGraphDef meta data that precise the way input is formatted and presented to the model and the way output is sent. Also, SavedModel can contains any extra files required by your model (i.e lookup table(s), etc..).

This is the Python training task packaged in a .gz format. It is NOT used in serving the model on the cloud.

You can donwload the model to your local machine:

You can also examine the graph using TensorBoard

Now that we have trained the model we are ready to create the ML Engine Model:

Previous command will create an empty object. To "fill" it with actual model, we need to assign a storage address at which the actual model can be loaded. As we have mentioned before this corresponds to the SavedModel export directory:

From now on, ML Engine Model is active and operational. In production conditions, it is so often that the model is developed in multiple version. It is possible to create multiple model versions under the same model name. In the previous command we have created version 1. At later stages we can retrain the model using the same or different data set and create version 2 and so on. Each new version is saved under a different timestamp directory under the "export" directory according to: ./export/[model_name]/[time_stamp]/. In our example the timestamp is 1524259030.

Next, let us try to test the online model:

We should get an output similar to:

Even that the returned output contains multiple outputs it actually reflects a single output prediction value. For this response, have 0.0643 as the predicted probability. In our model it is defined as the logistic regression probability. LOGITS is the odd logarithm, defined as ln(p/(1-p)), [-2.676 = ln(0.0643/(1-0.0643)] . 0.935 is simply the "untrue" probability which is equal to 1-0.0643.

Online testing

We have hosted an instance of the model on our server. You can test the developed model online using this interface.