Simple Usage Question

[datistiquo]

In the getting started docs is nothing about how to use/upload my own code (for training or HP Tuning). It is only shown how to download a model form github. How do I get my code on the cloud? Why do I find anything related to that?

I want to do same like here for my own model and doing predictions and HP Tuning: https://github.com/GoogleCloudPlatform/cloudml-samples/blob/master/notebooks/tensorflow/getting-started-keras.ipynb

But I don't know how to upload my code.

[datistiquo]

How would I upload this

1) GUI via Storgae (?!) 2) or via gc console?

[gogasca]

Hi @datistiquo please follow this tutorial, basically you only need a trainer folder using the correct structure and then run thegcloud ai platform command to upload it and then execute the code in GCP. Example:

https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/census/keras (Although is Keras applies to TF as well)

[datistiquo]

So the only possible way is via SDK? I did not installed this yet.

If I use the commandsgcloud ai platform inside google cloud console(?!) it needs the same stucture like lokal (the folder names like trainer...). Why isn't there an upload button? I thought I could upload all the files and run gcloud ai platformwithin cloud console... It is so annoying and unintuitiv...Why is this so complicated and why is SDK only for python 2.7? What if I need python 3 locally? So I cannot upload anything?

[datistiquo]

Whichgcloud ai platform command you actually mean for uploading?

and then execute the code in GCP.

How?

I installed SDK but I suppose you need to do all the gcloud commands inside a python environment? I tested it in a python environment but gcloud command is not found (i am on win 10)

I struggle now for days. It s so confusing because you have multiple different/redundant text information on the GCP websites. I cannot find anything which is a simple intro for every IDIOT. I mean really Idiot. I miss everytime the exact environment (where I am? Am I local or on the cloud console?). Do I need to do all commands from my local console or cloud console? Because I started with that here: https://cloud.google.com/ml-engine/docs/tensorflow/getting-started-keras?hl=de and there I did all commands on the cloud console but it is also mentioning the SDK so why actually?

[datistiquo]

So, end of the day GCP works only for notebooks?

1) I tried this tut: https://github.com/GoogleCloudPlatform/cloudml-samples/blob/master/notebooks/tensorflow/getting-started-keras.ipynb

with colab. I uploaded va GUI manually the trainer folder on the bucket. But the working dir of colab notebook seems not to be mine?! How can I switch to my trainer folder to start training via notebook?

How do run this: https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/census/keras ?

What are the steps to run

JOB_DIR=census_keras
TRAIN_STEPS=200
gcloud ai-platform local train --package-path trainer \
                             --module-name trainer.task \
                             -- \
                             --train-files $TRAIN_FILE \
                             --eval-files $EVAL_FILE \
                             --job-dir $JOB_DIR \
                             --train-steps $TRAIN_STEPS

At which location I have to be for that?

Where I am here at this point from first link:

Part 1. Quickstart for training in AI Platform This section of the tutorial walks you through submitting a training job to Cloud AI Platform. This job runs sample code that uses Keras to train a deep neural network on the United States Census data. It outputs the trained model as a TensorFlow SavedModel directory in your Cloud Storage bucket.

Get training code and dependencies First, download the training code and change the notebook's working directory:

Am I inside the created bucket? How can I change to the folder created via the GUI via colab notebook? Every doc is about downloading from github but not starting with uploading code from local. It is so desperate.

[nnegrey]

Hi @datistiquo, this is a repository comprised of samples for how to run code on AI Platform.

To run these samples you will have to clone this repository to your local machine: git clone https://github.com/GoogleCloudPlatform/cloudml-samples.git Then follow the steps provided in the README of a sample to complete the guide.

As pointed out above by gogasca, you need to download the Google Cloud SDK https://cloud.google.com/sdk/docs/quickstart-windows

The code is then put inside a package and uploaded to AI Platform training for you by the gcloud sdk and an output GCS bucket is where the model is stored.