search

canonical / kfp-operators

Kubeflow Pipelines Operators
Apache License 2.0
2 stars 12 forks source link

kfp-viz and kfp-ui seem to be partially ignoring OCI images provided as juju resources #410

Closed gustavosr98 closed 4 months ago

gustavosr98 commented 7 months ago

Bug Description

Deploying on an airgapped environment following procedure like this https://github.com/canonical/bundle-kubeflow/blob/main/releases/1.7/edge/kubeflow/airgap/podspec_script.sh#L12

Juju status reports to be okay But I see a couple of pods still trying to pull image out to the Internet rather than using the ones provided for the private OCI registry

To Reproduce

Follow this https://github.com/canonical/bundle-kubeflow/blob/main/releases/1.7/edge/kubeflow/airgap/podspec_script.sh#L12

Environment

Juju 3.1 Kubeflow 1.7 Microk8s 1.28-strict AirGapped

Charm versions

ll ~/charms/kubeflow/17/
total 426370
drwxrwxr-x 2 ubuntu ubuntu       42 Mar  4 02:51 ./
drwxrwxr-x 6 ubuntu ubuntu        6 Feb 28 00:31 ../
-rw-rw-r-- 1 ubuntu ubuntu  7291659 Feb 28 00:28 admission-webhook__1.7stable__rev224
-rw-rw-r-- 1 ubuntu ubuntu  8864108 Feb 28 00:28 argo-controller__3.3stable__rev376
-rw-rw-r-- 1 ubuntu ubuntu  7295116 Feb 28 00:29 argo-server__3.3stable__rev309
-rw-rw-r-- 1 ubuntu ubuntu     1292 Feb 28 01:01 charm_lists.txt
-rw-rw-r-- 1 ubuntu ubuntu 11803578 Feb 28 00:29 dex-auth__2.31stable__rev389
-rw-rw-r-- 1 ubuntu ubuntu 10707358 Feb 28 00:29 istio-gateway__1.16stable__rev663
-rw-rw-r-- 1 ubuntu ubuntu 37487674 Feb 28 00:29 istio-pilot__1.16stable__rev662
-rw-rw-r-- 1 ubuntu ubuntu  7366345 Feb 28 00:29 jupyter-controller__1.7stable__rev805
-rw-rw-r-- 1 ubuntu ubuntu 11623333 Feb 28 00:29 jupyter-ui__1.7stable__rev739
-rw-rw-r-- 1 ubuntu ubuntu  8067164 Feb 28 01:01 katib-controller__0.15edge__rev471
-rw-rw-r-- 1 ubuntu ubuntu  7565537 Feb 28 00:29 katib-controller__0.15stable__rev282
-rw-rw-r-- 1 ubuntu ubuntu 10904545 Feb 28 01:01 katib-db-manager__0.15edge__rev435
-rw-rw-r-- 1 ubuntu ubuntu 10741028 Feb 28 00:29 katib-db-manager__0.15stable__rev253
-rw-rw-r-- 1 ubuntu ubuntu 12012493 Feb 28 01:01 katib-ui__0.15edge__rev446
-rw-rw-r-- 1 ubuntu ubuntu 11871018 Feb 28 00:29 katib-ui__0.15stable__rev267
-rw-rw-r-- 1 ubuntu ubuntu 11876193 Feb 28 00:29 kfp-api__2.0-alpha.7stable__rev935
-rw-rw-r-- 1 ubuntu ubuntu  9279181 Feb 28 00:29 kfp-persistence__2.0-alpha.7stable__rev939
-rw-rw-r-- 1 ubuntu ubuntu  9286049 Feb 28 00:29 kfp-profile-controller__2.0-alpha.7stable__rev899
-rw-rw-r-- 1 ubuntu ubuntu  7355504 Feb 28 00:30 kfp-schedwf__2.0-alpha.7stable__rev952
-rw-rw-r-- 1 ubuntu ubuntu  9547670 Feb 28 00:30 kfp-ui__2.0-alpha.7stable__rev934
-rw-rw-r-- 1 ubuntu ubuntu  7355434 Feb 28 00:30 kfp-viewer__2.0-alpha.7stable__rev964
-rw-rw-r-- 1 ubuntu ubuntu  9546079 Feb 28 00:30 kfp-viz__2.0-alpha.7stable__rev889
-rw-rw-r-- 1 ubuntu ubuntu 10524672 Feb 28 00:30 knative-eventing__1.8stable__rev345
-rw-rw-r-- 1 ubuntu ubuntu 10586233 Feb 28 00:30 knative-operator__1.8stable__rev320
-rw-rw-r-- 1 ubuntu ubuntu 10528503 Feb 28 00:30 knative-serving__1.8stable__rev346
-rw-rw-r-- 1 ubuntu ubuntu 10636257 Feb 28 00:30 kserve-controller__0.10stable__rev458
-rw-rw-r-- 1 ubuntu ubuntu 11535259 Feb 28 00:30 kubeflow-dashboard__1.7stable__rev439
-rw-rw-r-- 1 ubuntu ubuntu 11572362 Feb 28 00:30 kubeflow-profiles__1.7stable__rev336
-rw-rw-r-- 1 ubuntu ubuntu  8847100 Feb 28 00:30 kubeflow-roles__1.7stable__rev148
-rw-rw-r-- 1 ubuntu ubuntu  8794888 Feb 28 00:30 kubeflow-volumes__1.7stable__rev204
-rw-rw-r-- 1 ubuntu ubuntu 15038613 Feb 28 00:31 metacontroller-operator__2.0stable__rev204
-rw-rw-r-- 1 ubuntu ubuntu  8847909 Feb 28 00:31 minio__ckf-1.7stable__rev214
-rw-rw-r-- 1 ubuntu ubuntu 24133839 Mar  4 02:32 mlflow-server__2.1stable__rev466
-rw-rw-r-- 1 ubuntu ubuntu 30360738 Feb 28 00:31 mysql-k8s__8.0stable__rev113
-rw-rw-r-- 1 ubuntu ubuntu  4419824 Feb 28 00:31 oidc-gatekeeper__ckf-1.7stable__rev320
-rw-rw-r-- 1 ubuntu ubuntu 11786019 Mar  4 02:51 resource-dispatcher__1.0stable__rev93
-rw-rw-r-- 1 ubuntu ubuntu 10674029 Feb 28 00:31 seldon-core__1.15stable__rev548
-rw-rw-r-- 1 ubuntu ubuntu  7279252 Feb 28 00:31 tensorboard-controller__1.7stable__rev156
-rw-rw-r-- 1 ubuntu ubuntu  8682275 Feb 28 00:31 tensorboards-web-app__1.7stable__rev158
-rw-rw-r-- 1 ubuntu ubuntu 10788445 Feb 28 00:31 training-operator__1.6stable__rev305

OCI images

# cat images_kubeflow_17.txt
docker.io/kubeflownotebookswg/poddefaults-webhook:v1.7.0
argoproj/workflow-controller:v3.3.9
argoproj/argocli:v3.3.9
charmedkubeflow/argoexec:v3.3.9_22.04_2
charmedkubeflow/dex:2.31.2_b2b609b
docker.io/istio/pilot:1.16.2
docker.io/istio/proxyv2:1.16.2
docker.io/kubeflowkatib/katib-db-manager:v0.15.0
docker.io/kubeflowkatib/katib-controller:v0.15.0
docker.io/kubeflowkatib/katib-ui:v0.15.0
docker.io/ubuntu:22.04
docker.io/kubeflowkatib/earlystopping-medianstop:v0.15.0
docker.io/kubeflowkatib/enas-cnn-cifar10-cpu:v0.15.0
docker.io/kubeflowkatib/file-metrics-collector:v0.15.0
docker.io/kubeflowkatib/file-metrics-collector:v0.15.0
docker.io/kubeflowkatib/tfevent-metrics-collector:v0.15.0
docker.io/kubeflowkatib/pytorch-mnist-cpu:v0.15.0
docker.io/kubeflowkatib/pytorch-mnist-cpu:v0.15.0
docker.io/kubeflowkatib/suggestion-hyperopt:v0.15.0
docker.io/kubeflowkatib/suggestion-hyperopt:v0.15.0
docker.io/kubeflowkatib/suggestion-optuna:v0.15.0
docker.io/kubeflowkatib/suggestion-hyperband:v0.15.0
docker.io/kubeflowkatib/suggestion-skopt:v0.15.0
docker.io/kubeflowkatib/suggestion-goptuna:v0.15.0
docker.io/kubeflowkatib/suggestion-goptuna:v0.15.0
docker.io/kubeflowkatib/suggestion-optuna:v0.15.0
docker.io/kubeflowkatib/suggestion-enas:v0.15.0
docker.io/kubeflowkatib/suggestion-darts:v0.15.0
docker.io/kubeflowkatib/suggestion-pbt:v0.15.0
charmedkubeflow/persistenceagent:2.0.0-alpha.7_22.04_1
charmedkubeflow/api-server:2.0.0-alpha.7_20.04_1
charmedkubeflow/scheduledworkflow:2.0.0-alpha.7_22.04_2
gcr.io/ml-pipeline/frontend:2.0.0-alpha.7
charmedkubeflow/visualization-server:2.0.0-alpha.7_20.04_1
python:3.7
charmedkubeflow/viewer-crd-controller:2.0.0-alpha.7_22.04_1
gcr.io/google-containers/busybox
gcr.io/knative-releases/knative.dev/eventing/cmd/broker/filter@sha256:33ea8a657b974d7bf3d94c0b601a4fc287c1fb33430b3dda028a1a189e3d9526
gcr.io/knative-releases/knative.dev/eventing/cmd/broker/ingress@sha256:f4a9dfce9eec5272c90a19dbdf791fffc98bc5a6649ee85cb8a29bd5145635b1
gcr.io/knative-releases/knative.dev/eventing/cmd/controller@sha256:cbc452f35842cc8a78240642adc1ebb11a4c4d7c143c8277edb49012f6cfc5d3
gcr.io/knative-releases/knative.dev/eventing/cmd/in_memory/channel_controller@sha256:3ced549336c7ccf3bb2adf23a558eb55bd1aec7be17837062d21c749dfce8ce5
gcr.io/knative-releases/knative.dev/eventing/cmd/in_memory/channel_dispatcher@sha256:e17bbdf951868359424cd0a0465da8ef44c66ba7111292444ce555c83e280f1a
gcr.io/knative-releases/knative.dev/eventing/cmd/mtchannel_broker@sha256:c5d3664780b394f6d3e546eb94c972965fbd9357da5e442c66455db7ca94124c
gcr.io/knative-releases/knative.dev/eventing/cmd/mtping@sha256:bc200a12cbad35bea51aabe800a365f28a5bd1dd65b3934b3db2e7e22df37efd
gcr.io/knative-releases/knative.dev/eventing/cmd/webhook@sha256:c9c582f530155d22c01b43957ae0dba549b1cc903f77ec6cc1acb9ae9085be62
gcr.io/knative-releases/knative.dev/pkg/apiextensions/storageversion/cmd/migrate@sha256:59431cf8337532edcd9a4bcd030591866cc867f13bee875d81757c960a53668d
gcr.io/knative-releases/knative.dev/pkg/apiextensions/storageversion/cmd/migrate@sha256:d0095787bc1687e2d8180b36a66997733a52f8c49c3e7751f067813e3fb54b66
gcr.io/knative-releases/knative.dev/serving/cmd/activator@sha256:c3bbf3a96920048869dcab8e133e00f59855670b8a0bbca3d72ced2f512eb5e1
gcr.io/knative-releases/knative.dev/serving/cmd/autoscaler-hpa@sha256:7003443f0faabbaca12249aa16b73fa171bddf350abd826dd93b06f5080a146d
gcr.io/knative-releases/knative.dev/serving/cmd/autoscaler@sha256:caae5e34b4cb311ed8551f2778cfca566a77a924a59b775bd516fa8b5e3c1d7f
gcr.io/knative-releases/knative.dev/serving/cmd/controller@sha256:38f9557f4d61ec79cc2cdbe76da8df6c6ae5f978a50a2847c22cc61aa240da95
gcr.io/knative-releases/knative.dev/serving/cmd/default-domain@sha256:2b3e4074f951dd6b6fa5018047e97a5cb10b6a6ecbb554ed77c9a8dc01011f3b
gcr.io/knative-releases/knative.dev/serving/cmd/domain-mapping-webhook@sha256:a4ba0076df2efaca2eed561339e21b3a4ca9d90167befd31de882bff69639470
gcr.io/knative-releases/knative.dev/serving/cmd/domain-mapping@sha256:763d648bf1edee2b4471b0e211dbc53ba2d28f92e4dae28ccd39af7185ef2c96
gcr.io/knative-releases/knative.dev/serving/cmd/queue@sha256:505179c0c4892ea4a70e78bc52ac21b03cd7f1a763d2ecc78e7bbaa1ae59c86c
gcr.io/knative-releases/knative.dev/serving/cmd/webhook@sha256:bc13765ba4895c0fa318a065392d05d0adc0e20415c739e0aacb3f56140bf9ae
gcr.io/knative-releases/knative.dev/net-istio/cmd/controller@sha256:2b484d982ef1a5d6ff93c46d3e45f51c2605c2e3ed766e20247d1727eb5ce918
gcr.io/knative-releases/knative.dev/net-istio/cmd/webhook@sha256:59b6a46d3b55a03507c76a3afe8a4ee5f1a38f1130fd3d65c9fe57fff583fa8d
gcr.io/knative-releases/knative.dev/operator/cmd/operator:v1.8.1
gcr.io/knative-releases/knative.dev/operator/cmd/webhook:v1.8.1
otel/opentelemetry-collector:latest
kserve/kserve-controller:v0.10.0
gcr.io/kubebuilder/kube-rbac-proxy:v0.10.0
kserve/agent:v0.10.0
kserve/agent:v0.10.0
kserve/alibi-explainer:latest
ubuntu:22.04
kserve/art-explainer:latest
kserve/agent:v0.10.0
kserve/router:v0.10.0
charmedkubeflow/storage-initializer:0.10.0-252057c
kserve/lgbserver:v0.10.0
docker.io/seldonio/mlserver:1.3.5
kserve/paddleserver:v0.10.0
charmedkubeflow/pmmlserver:v0.10.0_22.04_1
kserve/sklearnserver:v0.10.0
tensorflow/serving:2.6.2
pytorch/torchserve-kfs:0.7.0
nvcr.io/nvidia/tritonserver:21.09-py3
kserve/xgbserver:v0.10.0
docker.io/kubeflownotebookswg/centraldashboard:v1.7.0
docker.io/kubeflownotebookswg/profile-controller:v1.7.0
docker.io/kubeflownotebookswg/kfam:v1.7.0
kubeflownotebookswg/tensorboard-controller:v1.7.0
kubeflownotebookswg/tensorboards-web-app:v1.7.0
docker.io/kubeflownotebookswg/volumes-web-app:v1.7.0
charmedkubeflow/metacontroller:2.0.4_22.04_1
minio/minio:RELEASE.2021-09-03T03-56-13Z
docker.io/kubeflownotebookswg/notebook-controller:v1.7.0
docker.io/kubeflownotebookswg/jupyter-web-app:v1.7.0
charmedkubeflow/jupyter-scipy:v1.7.0-20.04-1_ad4e002
charmedkubeflow/jupyter-pytorch-full:v1.7.0-20.04-1_ad4e002
charmedkubeflow/jupyter-pytorch-cuda-full:v1.7.0-20.04-1_ad4e002
charmedkubeflow/jupyter-tensorflow-full:v1.7.0-20.04-1_ad4e002
charmedkubeflow/jupyter-tensorflow-cuda-full:v1.7.0-20.04-1_ad4e002
kubeflownotebookswg/codeserver-python:v1.7.0
kubeflownotebookswg/rstudio-tidyverse:v1.7.0
charmedkubeflow/oidc-authservice:ckf-1.7_9517371
docker.io/charmedkubeflow/seldon-core-operator:v1.15.0_22.04_1
docker.io/seldonio/seldon-core-executor:1.14.0
tensorflow/serving:2.1.0
seldonio/tfserving-proxy:1.15.0
docker.io/charmedkubeflow/sklearnserver:v1.16.0_20.04_1
docker.io/charmedkubeflow/mlserver-sklearn:1.2.0_22.04_1
seldonio/xgboostserver:1.15.0
docker.io/charmedkubeflow/mlserver-xgboost:1.2.0_22.04_1
seldonio/mlflowserver:1.15.0
docker.io/charmedkubeflow/mlserver-mlflow:1.2.0_22.04_1
nvcr.io/nvidia/tritonserver:21.08-py3
docker.io/charmedkubeflow/mlserver-huggingface:1.2.4_22.04_1
seldonio/mlserver:1.2.0-slim
seldonio/rclone-storage-initializer:1.14.1
seldonio/alibiexplainer:1.15.0
seldonio/mlserver:1.2.0-alibi-explain
kubeflow/training-operator:v1-66aa635
ghcr.io/canonical/charmed-mysql@sha256:a0aad90152a77767569840cfd76eb957a1d26f43dc8d3922be823506e7533186
docker.io/ubuntu/mlflow:2.1.1_1.0-22.04
docker.io/charmedkubeflow/mlflow-prometheus-exporter:1.0-22.04
charmedkubeflow/resource-dispatcher:1.0-22.04

Relevant Log Output

# juju status
Model     Controller  Cloud/Region              Version  SLA          Timestamp
kubeflow  lxd-mgmt    microk8s-train/localhost  3.1.7    unsupported  05:52:14Z

App                      Version                         Status  Scale  Charm                    Channel  Rev  Address         Exposed  Message
admission-webhook        .../kubeflownotebookswg/pod...  active      1  admission-webhook                   0  10.152.183.166  no
argo-controller          .../workflow-controller:v3.3.9  active      1  argo-controller                     0                  no
argo-server              argoproj/argocli:v3.3.9         active      1  argo-server                         0                  no
dex-auth                                                 active      1  dex-auth                            0  10.152.183.148  no
istio-ingressgateway                                     active      1  istio-gateway                       0  10.152.183.46   no
istio-pilot                                              active      1  istio-pilot                         0  10.152.183.172  no
jupyter-controller       .../kubeflownotebookswg/not...  active      1  jupyter-controller                  0                  no
jupyter-ui                                               active      1  jupyter-ui                          0  10.152.183.68   no
katib-controller         .../kubeflowkatib/katib-con...  active      1  katib-controller                    0  10.152.183.241  no
katib-db                 8.0.35-0ubuntu0.22.04.1         active      1  mysql-k8s                           1  10.152.183.25   no
katib-db-manager                                         active      1  katib-db-manager                    0  10.152.183.152  no
katib-ui                                                 active      1  katib-ui                            0  10.152.183.57   no
kfp-api                                                  active      1  kfp-api                             0  10.152.183.202  no
kfp-db                   8.0.35-0ubuntu0.22.04.1         active      1  mysql-k8s                           0  10.152.183.170  no
kfp-persistence          .../persistenceagent:2.0.0-...  active      1  kfp-persistence                     0                  no
kfp-profile-controller   python:3.7                      active      1  kfp-profile-controller              0  10.152.183.253  no
kfp-schedwf              .../scheduledworkflow:2.0.0...  active      1  kfp-schedwf                         0                  no
kfp-ui                   .../ml-pipeline/frontend:2....  active      1  kfp-ui                              0  10.152.183.205  no
kfp-viewer               .../viewer-crd-controller:2...  active      1  kfp-viewer                          0                  no
kfp-viz                  .../visualization-server:2....  active      1  kfp-viz                             0  10.152.183.130  no
kubeflow-dashboard                                       active      1  kubeflow-dashboard                  0  10.152.183.254  no
kubeflow-profiles                                        active      1  kubeflow-profiles                   0  10.152.183.156  no
kubeflow-roles                                           active      1  kubeflow-roles                      0  10.152.183.235  no
kubeflow-volumes         .../kubeflownotebookswg/vol...  active      1  kubeflow-volumes                    0  10.152.183.212  no
metacontroller-operator                                  active      1  metacontroller-operator             0  10.152.183.195  no
minio                    .../minio:RELEASE.2021-09-0...  active      1  minio                               0  10.152.183.159  no
mlflow-mysql             8.0.35-0ubuntu0.22.04.1         active      1  mysql-k8s                           2  10.152.183.245  no
mlflow-server                                            active      1  mlflow-server                       1  10.152.183.89   no
oidc-gatekeeper          .../oidc-authservice:ckf-1....  active      1  oidc-gatekeeper                     0  10.152.183.97   no
resource-dispatcher                                      active      1  resource-dispatcher                 0  10.152.183.177  no
tensorboard-controller   .../tensorboard-controller:...  active      1  tensorboard-controller              0  10.152.183.179  no
tensorboards-web-app     .../tensorboards-web-app:v1...  active      1  tensorboards-web-app                0  10.152.183.92   no
training-operator                                        active      1  training-operator                   0  10.152.183.88   no

Unit                        Workload  Agent  Address       Ports          Message
admission-webhook/0*        active    idle   10.1.195.226  4443/TCP
argo-controller/0*          active    idle   10.1.195.235
argo-server/0*              active    idle   10.1.195.230  2746/TCP
dex-auth/0*                 active    idle   10.1.195.236
istio-ingressgateway/0*     active    idle   10.1.195.199
istio-pilot/0*              active    idle   10.1.195.248
jupyter-controller/0*       active    idle   10.1.195.244
jupyter-ui/0*               active    idle   10.1.195.250
katib-controller/0*         active    idle   10.1.195.202  443,8080/TCP
katib-db-manager/0*         active    idle   10.1.195.218
katib-db/0*                 active    idle   10.1.195.207                 Primary
katib-ui/0*                 active    idle   10.1.195.228
kfp-api/0*                  active    idle   10.1.195.225
kfp-db/0*                   active    idle   10.1.195.220                 Primary
kfp-persistence/0*          active    idle   10.1.195.208
kfp-profile-controller/0*   active    idle   10.1.195.241  80/TCP
kfp-schedwf/0*              active    idle   10.1.195.249
kfp-ui/0*                   active    idle   10.1.195.243  3000/TCP
kfp-viewer/0*               active    idle   10.1.195.216
kfp-viz/0*                  active    idle   10.1.195.234  8888/TCP
kubeflow-dashboard/0*       active    idle   10.1.195.223
kubeflow-profiles/0*        active    idle   10.1.195.239
kubeflow-roles/0*           active    idle   10.1.195.240
kubeflow-volumes/0*         active    idle   10.1.195.227  5000/TCP
metacontroller-operator/0*  active    idle   10.1.195.210
minio/0*                    active    idle   10.1.195.204  9000-9001/TCP
mlflow-mysql/0*             active    idle   10.1.195.255                 Primary
mlflow-server/0*            active    idle   10.1.195.198
oidc-gatekeeper/0*          active    idle   10.1.195.221  8080/TCP
resource-dispatcher/0*      active    idle   10.1.195.254
tensorboard-controller/0*   active    idle   10.1.195.201  9443/TCP
tensorboards-web-app/0*     active    idle   10.1.195.209  5000/TCP
training-operator/0*        active    idle   10.1.195.233

# juju status | grep -v active
Model     Controller  Cloud/Region              Version  SLA          Timestamp
kubeflow  lxd-mgmt    microk8s-train/localhost  3.1.7    unsupported  05:44:25Z

App                      Version                         Status  Scale  Charm                    Channel  Rev  Address         Exposed  Message

Unit                        Workload  Agent  Address       Ports          Message

# ubuntu@mltr01:~$ microk8s.kubectl get po -n admin | grep -v Running
NAME                                              READY   STATUS             RESTARTS   AGE
ml-pipeline-visualizationserver-9d8bfdc78-52k4s   1/2     ImagePullBackOff   0          135m
ml-pipeline-ui-artifact-6cfb9c4886-zn8rp          1/2     ImagePullBackOff   0          135m

# microk8s.kubectl describe -n admin po ml-pipeline-visualizationserver-9d8bfdc78-52k4s
  Normal  BackOff  42s (x535 over 135m)  kubelet  Back-off pulling image "gcr.io/ml-pipeline/visualization-server:2.0.0-alpha.7"

# microk8s.kubectl describe -n admin po ml-pipeline-ui-artifact-6cfb9c4886-zn8rp
  Normal  BackOff  85s (x539 over 135m)  kubelet  Back-off pulling image "gcr.io/ml-pipeline/frontend:2.0.0-alpha.7"

# sudo docker image list | grep 10.10.11 | grep -E "visualization|ml-pipeline/frontend"
10.10.11.39:32000/charmedkubeflow/visualization-server                                               2.0.3-8169d0c                  b4d40d3b0e2e   2 months ago    3.72GB
10.10.11.39:32000/charmedkubeflow/visualization-server                                               2.0.0-alpha.7_20.04_1          855dd810ff53   7 months ago    3.09GB
10.10.11.39:32000/gcr.io/ml-pipeline/frontend                                                        2.0.0-alpha.7                  d88e95da69b6   12 months ago   513MB

Additional Context

No response

syncronize-issues-to-jira[bot] commented 7 months ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5418.

This message was autogenerated

NohaIhab commented 4 months ago

This is now working in CKF 1.8. Tested with the deployment script merged in https://github.com/canonical/bundle-kubeflow/pull/908. @gustavosr98 If you deploy CKF 1.8 with the image configs defined in 1.8 deployment script, you should no longer see this issue.