search

kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.61k stars 1.63k forks source link

Standalone deployment is not using the proxy in the correct region #4284

Closed numerology closed 4 years ago

numerology commented 4 years ago

If a standalone deployment is deployed to a us-central1 GKE cluster, it is supposed to talk to inverse proxy us-central1. However currently it seems like it's talking to us-central2, which is inconsistent with our proxy config.

numerology commented 4 years ago

/assign @rmgogogo /assign @numerology

/cc @abdallag

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Bobgy commented 4 years ago

@numerology can you provide more details?

Choosing which proxy is code we maintain in KFP, so one of us need to investigate this.

Bobgy commented 4 years ago

https://github.com/kubeflow/pipelines/blob/0ff6d9cfa438119785b94b8802f4bf7d36f95bda/proxy/attempt-register-vm-on-proxy.sh

numerology commented 4 years ago

/lifecycle frozen

numerology commented 4 years ago

@Bobgy Sorry I dropped the ball on this one...

For example, if a user deploys KFP onto a GKE cluster in us-central1 region, then according to this config inverse proxy should return a host name suffixed by us-central1, but it's currently xxx-us-central2. It's not blocking usage but suboptimal.

Bobgy commented 4 years ago

Understood, FYI code for deciding which proxy url to choose is in https://github.com/kubeflow/pipelines/blob/0ff6d9cfa438119785b94b8802f4bf7d36f95bda/proxy/get_proxy_url.py, so anyone who'll try to fix this problem should troubleshoot there.

suryaavala commented 4 years ago

@Bobgy I submitted a fix in #4702, would you be able to review it?