search

canonical / bundle-kubeflow

Charmed Kubeflow
Apache License 2.0
102 stars 49 forks source link

tensorboard-controller issue KF-1.6 #488

Closed aym-frikha closed 1 year ago

aym-frikha commented 2 years ago

Hello,

Trying to deploy charmed KF 1.6 beta on microk8s 1.22. I had an issue with tensorboard-controller wich is stuck on "Waiting for gateway relation". Here is the logs for the unit.

DnPlas commented 2 years ago

Hi @aym-frikha that is a known issue that has to do with how istio-pilot manages the gateway-relation. We have a fix already merged in the 1.6 track and you should be able to see that message disappear if you deploy any Charmed Kubeflow with istio-pilot rev above 126. This issue will be fixed by the 1.6/stable release.

ca-scribner commented 2 years ago

I believe this is now fixed and pushed to 1.6/stable, but please reopen if I misunderstood

ca-scribner commented 1 year ago

This is still sometimes an issue. For mitigation steps, see the Kubeflow Dashboard can’t be accessed section of this quickstart guide

ca-scribner commented 1 year ago

Progress on this issue is tracked in canonical/istio-operators#113

ShathishWarmaS commented 1 year ago

tensorboard-controller waiting 1 tensorboard-controller 1.7/stable 156 no Waiting for gateway info relation

Its been solved by microk8s kubectl get gateway -A juju run --unit istio-pilot/0 -- "export JUJU_DISPATCH_PATH=hooks/config-changed; ./dispatch" nice -n 16 watch -n 1 -c juju status --relations --color