Building-ML-Pipelines / building-machine-learning-pipelines

Code repository for the O'Reilly publication "Building Machine Learning Pipelines" by Hannes Hapke & Catherine Nelson
MIT License
584 stars 249 forks source link

Image not publicly visible in gcr.io container registry #29

Closed mshearer0 closed 2 years ago

mshearer0 commented 4 years ago

Error pulling image /oreilly-book/ml-pipelines-tfx-custom:0.22.0 from container registry gcr.io. Is it globally visible?

mshearer0 commented 4 years ago

Building and deploying my own custom image with tensorflow-hub following instructions on pg 317,318 allows kubeflow pipeline to complete

Updating pipeline_kubeflow_gcp_buckets.py:

input_bucket and output_bucket to point to my cloud storage bucket (and copying over data and components folders to that location).

tfx_image = os.environ.get( "KUBEFLOW_TFX_IMAGE", "gcr.io/insert your project_id/ml-pipelines-tfx-custom:0.22.0",

hanneshapke commented 4 years ago

Thank you @mshearer0 You raise a good point. I'll make the gcr url dynamic.

hanneshapke commented 4 years ago

@mshearer0 If you have a moment, please check out https://github.com/Building-ML-Pipelines/building-machine-learning-pipelines/pull/31 and let me know if the implementation would work for you.

mshearer0 commented 4 years ago

Thanks, should pipeline.py also be updated

And

tfx_image = os.environ.get( "KUBEFLOW_TFX_IMAGE", "gcr.io/oreilly-book/ml-pipelines-tfx-custom:0.21.4", )

In pipeline_gcp_cloud_ai.py?

jazzsir commented 3 years ago

@hanneshapke #31 does not work for me, could you tell me a proper image path?

mshearer0 commented 3 years ago

@jazzsir - did you build and deploy your own image into gcr.io?

jazzsir commented 3 years ago

@mshearer0 I finally applied my custom image to my own registry server, thanks

jazzsir commented 3 years ago

@hanneshapke did you run kubeflow example code without any errors at all? my case #36

hanneshapke commented 2 years ago

Hi @jazzsir,

Yes, it did. Could it be a permission issue to the Google Container Registry or GCS?

PS: Check out the latest updates to the example code: https://github.com/Building-ML-Pipelines/building-machine-learning-pipelines/releases/tag/examples_based_on_tfx_1.4