search

GoogleCloudPlatform / vertex-ai-samples

Notebooks, code samples, sample apps, and other resources that demonstrate how to use, develop and manage machine learning and generative AI workflows using Google Cloud Vertex AI.
https://cloud.google.com/vertex-ai
Apache License 2.0
1.65k stars 815 forks source link

Ray on Vertex Notebook Failure #3078

Open happy-qiao opened 3 months ago

happy-qiao commented 3 months ago

Expected Behavior

https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/ray_on_vertex_ai/get_started_with_pytorch_rov.ipynb run successfully

Actual Behavior

https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/ray_on_vertex_ai/get_started_with_pytorch_rov.ipynb failed at this cell

client = JobSubmissionClient(
    address="vertex_ray://{}".format(ray_cluster.dashboard_address)
)

with error message

Screenshot 2024-06-10 at 13 55 03

Steps to Reproduce the Problem

1.Run this script in workbench: Open in Vertex AI Workbench 1.Remove ! gcloud config set project {PROJECT_ID} because workbench already has project_id set. 1.Fill necessary params and execute All cells

Specifications

gericdong commented 3 months ago

@inardini can you please help take a look? Thanks

inardini commented 3 months ago

This is going to be address here #3131 . Thanks