search

canonical / bundle-kubeflow

Charmed Kubeflow
Apache License 2.0
97 stars 47 forks source link

Deployment of CKF 1.8 fails with Juju 3.1 #950

Closed ACodingfreak closed 16 hours ago

ACodingfreak commented 3 days ago

Bug Description

As shown in below logs on deploying 1.8/stable kubeflow from Juju 3.1 fails because of dependency issues of katib-db. I am just following the instructions suggested by canonical in below article

https://charmed-kubeflow.io/docs/get-started-with-charmed-kubeflow

mm321:~$ juju deploy kubeflow --trust  --channel=1.8/stable
Located bundle "kubeflow" in charm-hub, revision 414
Located charm "admission-webhook" in charm-hub, channel 1.8/stable
Located charm "argo-controller" in charm-hub, channel 3.3.10/stable
Located charm "dex-auth" in charm-hub, channel 2.36/stable
Located charm "envoy" in charm-hub, channel 2.0/stable
Located charm "istio-gateway" in charm-hub, channel 1.17/stable
Located charm "istio-pilot" in charm-hub, channel 1.17/stable
Located charm "jupyter-controller" in charm-hub, channel 1.8/stable
Located charm "jupyter-ui" in charm-hub, channel 1.8/stable
Located charm "katib-controller" in charm-hub, channel 0.16/stable
Located charm "mysql-k8s" in charm-hub, channel 8.0/stable
Located charm "katib-db-manager" in charm-hub, channel 0.16/stable
Located charm "katib-ui" in charm-hub, channel 0.16/stable
Located charm "kfp-api" in charm-hub, channel 2.0/stable
Located charm "mysql-k8s" in charm-hub, channel 8.0/stable
Located charm "kfp-metadata-writer" in charm-hub, channel 2.0/stable
Located charm "kfp-persistence" in charm-hub, channel 2.0/stable
Located charm "kfp-profile-controller" in charm-hub, channel 2.0/stable
Located charm "kfp-schedwf" in charm-hub, channel 2.0/stable
Located charm "kfp-ui" in charm-hub, channel 2.0/stable
Located charm "kfp-viewer" in charm-hub, channel 2.0/stable
Located charm "kfp-viz" in charm-hub, channel 2.0/stable
Located charm "knative-eventing" in charm-hub, channel 1.10/stable
Located charm "knative-operator" in charm-hub, channel 1.10/stable
Located charm "knative-serving" in charm-hub, channel 1.10/stable
Located charm "kserve-controller" in charm-hub, channel 0.11/stable
Located charm "kubeflow-dashboard" in charm-hub, channel 1.8/stable
Located charm "kubeflow-profiles" in charm-hub, channel 1.8/stable
Located charm "kubeflow-roles" in charm-hub, channel 1.8/stable
Located charm "kubeflow-volumes" in charm-hub, channel 1.8/stable
Located charm "metacontroller-operator" in charm-hub, channel 3.0/stable
Located charm "minio" in charm-hub, channel ckf-1.8/stable
Located charm "mlmd" in charm-hub, channel 1.14/stable
Located charm "oidc-gatekeeper" in charm-hub, channel ckf-1.8/stable
Located charm "pvcviewer-operator" in charm-hub, channel 1.8/stable
Located charm "seldon-core" in charm-hub, channel 1.17/stable
Located charm "tensorboard-controller" in charm-hub, channel 1.8/stable
Located charm "tensorboards-web-app" in charm-hub, channel 1.8/stable
Located charm "training-operator" in charm-hub, channel 1.7/stable
Executing changes:
- upload charm admission-webhook from charm-hub from channel 1.8/stable with architecture=amd64
- deploy application admission-webhook from charm-hub with 1 unit with 1.8/stable
  added resource oci-image
- upload charm argo-controller from charm-hub from channel 3.3.10/stable with architecture=amd64
- deploy application argo-controller from charm-hub with 1 unit with 3.3.10/stable
  added resource oci-image
- upload charm dex-auth from charm-hub from channel 2.36/stable with architecture=amd64
- deploy application dex-auth from charm-hub with 1 unit with 2.36/stable
  added resource oci-image
- upload charm envoy from charm-hub from channel 2.0/stable with architecture=amd64
- deploy application envoy from charm-hub with 1 unit with 2.0/stable
  added resource oci-image
- upload charm istio-gateway from charm-hub from channel 1.17/stable with architecture=amd64
- deploy application istio-ingressgateway from charm-hub with 1 unit with 1.17/stable using istio-gateway
- upload charm istio-pilot from charm-hub from channel 1.17/stable with architecture=amd64
- deploy application istio-pilot from charm-hub with 1 unit with 1.17/stable
- upload charm jupyter-controller from charm-hub from channel 1.8/stable with architecture=amd64
- deploy application jupyter-controller from charm-hub with 1 unit with 1.8/stable
  added resource oci-image
- upload charm jupyter-ui from charm-hub from channel 1.8/stable with architecture=amd64
- deploy application jupyter-ui from charm-hub with 1 unit with 1.8/stable
  added resource oci-image
- upload charm katib-controller from charm-hub from channel 0.16/stable with architecture=amd64
- deploy application katib-controller from charm-hub with 1 unit with 0.16/stable
  added resource oci-image
- upload charm mysql-k8s from charm-hub from channel 8.0/stable with architecture=amd64
- deploy application katib-db from charm-hub with 1 unit with 8.0/stable using mysql-k8s
ERROR cannot deploy bundle: cannot deploy application "katib-db": Charm feature requirements cannot be met:
  - charm requires at least one of the following:
    - charm requires all of the following:
      - charm requires feature "juju" (version < 3.0.0) but model currently supports version 3.1.8
    - charm requires all of the following:
      - charm requires feature "juju" (version >= 3.4.3) but model currently supports version 3.1.8
    - charm requires all of the following:
      - charm requires feature "juju" (version >= 3.5.2) but model currently supports version 3.1.8

Feature descriptions:
  - "juju": the version of Juju used by the model

For additional information please see: https://juju.is/docs/olm/supported-features

To Reproduce

Follow the instructions in below article

https://charmed-kubeflow.io/docs/get-started-with-charmed-kubeflow

sudo snap install microk8s --channel=1.29/stable --classic
sudo snap install juju --classic --channel=3.1/stable
microk8s config | juju add-k8s my-k8s --client
juju bootstrap my-k8s uk8sx
juju add-model kubeflow
juju deploy kubeflow --trust  --channel=1.8/stable

Environment

Ubuntu 22.04 microk8s 1.29/stable juju 3.1/stable kubeflow 1.8/stable

Relevant Log Output

mm321:~$ juju deploy kubeflow --trust  --channel=1.8/stable
Located bundle "kubeflow" in charm-hub, revision 414
Located charm "admission-webhook" in charm-hub, channel 1.8/stable
Located charm "argo-controller" in charm-hub, channel 3.3.10/stable
Located charm "dex-auth" in charm-hub, channel 2.36/stable
Located charm "envoy" in charm-hub, channel 2.0/stable
Located charm "istio-gateway" in charm-hub, channel 1.17/stable
Located charm "istio-pilot" in charm-hub, channel 1.17/stable
Located charm "jupyter-controller" in charm-hub, channel 1.8/stable
Located charm "jupyter-ui" in charm-hub, channel 1.8/stable
Located charm "katib-controller" in charm-hub, channel 0.16/stable
Located charm "mysql-k8s" in charm-hub, channel 8.0/stable
Located charm "katib-db-manager" in charm-hub, channel 0.16/stable
Located charm "katib-ui" in charm-hub, channel 0.16/stable
Located charm "kfp-api" in charm-hub, channel 2.0/stable
Located charm "mysql-k8s" in charm-hub, channel 8.0/stable
Located charm "kfp-metadata-writer" in charm-hub, channel 2.0/stable
Located charm "kfp-persistence" in charm-hub, channel 2.0/stable
Located charm "kfp-profile-controller" in charm-hub, channel 2.0/stable
Located charm "kfp-schedwf" in charm-hub, channel 2.0/stable
Located charm "kfp-ui" in charm-hub, channel 2.0/stable
Located charm "kfp-viewer" in charm-hub, channel 2.0/stable
Located charm "kfp-viz" in charm-hub, channel 2.0/stable
Located charm "knative-eventing" in charm-hub, channel 1.10/stable
Located charm "knative-operator" in charm-hub, channel 1.10/stable
Located charm "knative-serving" in charm-hub, channel 1.10/stable
Located charm "kserve-controller" in charm-hub, channel 0.11/stable
Located charm "kubeflow-dashboard" in charm-hub, channel 1.8/stable
Located charm "kubeflow-profiles" in charm-hub, channel 1.8/stable
Located charm "kubeflow-roles" in charm-hub, channel 1.8/stable
Located charm "kubeflow-volumes" in charm-hub, channel 1.8/stable
Located charm "metacontroller-operator" in charm-hub, channel 3.0/stable
Located charm "minio" in charm-hub, channel ckf-1.8/stable
Located charm "mlmd" in charm-hub, channel 1.14/stable
Located charm "oidc-gatekeeper" in charm-hub, channel ckf-1.8/stable
Located charm "pvcviewer-operator" in charm-hub, channel 1.8/stable
Located charm "seldon-core" in charm-hub, channel 1.17/stable
Located charm "tensorboard-controller" in charm-hub, channel 1.8/stable
Located charm "tensorboards-web-app" in charm-hub, channel 1.8/stable
Located charm "training-operator" in charm-hub, channel 1.7/stable
Executing changes:
- upload charm admission-webhook from charm-hub from channel 1.8/stable with architecture=amd64
- deploy application admission-webhook from charm-hub with 1 unit with 1.8/stable
  added resource oci-image
- upload charm argo-controller from charm-hub from channel 3.3.10/stable with architecture=amd64
- deploy application argo-controller from charm-hub with 1 unit with 3.3.10/stable
  added resource oci-image
- upload charm dex-auth from charm-hub from channel 2.36/stable with architecture=amd64
- deploy application dex-auth from charm-hub with 1 unit with 2.36/stable
  added resource oci-image
- upload charm envoy from charm-hub from channel 2.0/stable with architecture=amd64
- deploy application envoy from charm-hub with 1 unit with 2.0/stable
  added resource oci-image
- upload charm istio-gateway from charm-hub from channel 1.17/stable with architecture=amd64
- deploy application istio-ingressgateway from charm-hub with 1 unit with 1.17/stable using istio-gateway
- upload charm istio-pilot from charm-hub from channel 1.17/stable with architecture=amd64
- deploy application istio-pilot from charm-hub with 1 unit with 1.17/stable
- upload charm jupyter-controller from charm-hub from channel 1.8/stable with architecture=amd64
- deploy application jupyter-controller from charm-hub with 1 unit with 1.8/stable
  added resource oci-image
- upload charm jupyter-ui from charm-hub from channel 1.8/stable with architecture=amd64
- deploy application jupyter-ui from charm-hub with 1 unit with 1.8/stable
  added resource oci-image
- upload charm katib-controller from charm-hub from channel 0.16/stable with architecture=amd64
- deploy application katib-controller from charm-hub with 1 unit with 0.16/stable
  added resource oci-image
- upload charm mysql-k8s from charm-hub from channel 8.0/stable with architecture=amd64
- deploy application katib-db from charm-hub with 1 unit with 8.0/stable using mysql-k8s
ERROR cannot deploy bundle: cannot deploy application "katib-db": Charm feature requirements cannot be met:
  - charm requires at least one of the following:
    - charm requires all of the following:
      - charm requires feature "juju" (version < 3.0.0) but model currently supports version 3.1.8
    - charm requires all of the following:
      - charm requires feature "juju" (version >= 3.4.3) but model currently supports version 3.1.8
    - charm requires all of the following:
      - charm requires feature "juju" (version >= 3.5.2) but model currently supports version 3.1.8

Feature descriptions:
  - "juju": the version of Juju used by the model

For additional information please see: https://juju.is/docs/olm/supported-features



### Additional Context

_No response_
syncronize-issues-to-jira[bot] commented 3 days ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5913.

This message was autogenerated

brianmusisi commented 1 day ago

I run into the same issue and was able to resolve it by using Juju 3.4. After getting the same error, I run these steps to install Juju 3.4 and remove the controller and models created with Juju 3.1

Update to Juju 3.4

sudo snap refresh juju --channel 3.4/stable

Remove the old model and controller

juju destroy-model --destroy-storage --force kubeflow
juju destroy-controller --destroy-storage --force uk8sx

And then deploy the Juju controller and add the model again

juju bootstrap my-k8s uk8sx
juju add-model kubeflow

And finally, deploy kubeflow again

juju deploy kubeflow --trust  --channel=1.8/stable

I wasn't able to get this working with Juju 3.5 due to a similar error

ACodingfreak commented 1 day ago

Thanks for the reply @brianmusisi

I did try the same with Juju 3.4 microk8s 1.29 CKF 1.8 and ended up with katib-db-manager issue

https://github.com/canonical/bundle-kubeflow/issues/961

orfeas-k commented 16 hours ago

Hey, thank you for reporting this. Juju 3.1 is not supported anymore but we had not updated the documentation. We just did that prompting everyone to use Juju 3.4 (supported versions, get started guide). Closing this issue.