GoogleCloudPlatform / data-science-on-gcp

Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017
Apache License 2.0
1.32k stars 712 forks source link

Ch11 - error when running AutoML on full dataset #167

Closed jgammerman closed 1 year ago

jgammerman commented 1 year ago

Hi,

I've encountered an error when trying to run AutoML on the full dataset in chapter 11. My training data is in place, and I'm using the following command as per the textbook:

python3 train_on_vertexai.py --automl --project $PROJECT --bucket $BUCKET --region $REGION

My error is as follows:

2023-02-22 17:40:27.579468: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-02-22 17:40:27.579513: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
INFO:root:Training on []
Traceback (most recent call last):
  File "/home/jgammerman/data-science-on-gcp/11_realtime/train_on_vertexai.py", line 321, in <module>
    main()
  File "/home/jgammerman/data-science-on-gcp/11_realtime/train_on_vertexai.py", line 196, in main
    data_set = aiplatform.TabularDataset.create(
  File "/home/jgammerman/.local/lib/python3.9/site-packages/google/cloud/aiplatform/datasets/tabular_dataset.py", line 128, in create
    datasource = _datasources.create_datasource(
  File "/home/jgammerman/.local/lib/python3.9/site-packages/google/cloud/aiplatform/datasets/_datasources.py", line 224, in create_datasource
    return TabularDatasource(gcs_source, bq_source)
  File "/home/jgammerman/.local/lib/python3.9/site-packages/google/cloud/aiplatform/datasets/_datasources.py", line 86, in __init__
    raise ValueError("One of gcs_source or bq_source must be set.")
ValueError: One of gcs_source or bq_source must be set.

Where do I set gcs_source or bq_source?

jgammerman commented 1 year ago

UPDATE: found the bug, I had an error in the path to my bucket which I had needed to insert during a previous chapter.