search

canonical / charmed-kubeflow-uats

Automated UATs for Charmed Kubeflow
Apache License 2.0
5 stars 2 forks source link

update katib-integration.ipynb to use a new katib experiment #66

Closed ca-scribner closed 3 months ago

ca-scribner commented 3 months ago

This PR updates the katib example to include changes to the upstream example we based this test on. These changes were needed because the previous version requires a docker container that no longer exists.

Closes #64

Testing instructions:

I deployed a subset of Charmed Kubeflow using:

juju deploy istio-gateway --config kind=ingress --channel latest/edge --trust istio-ingressgateway
juju deploy istio-pilot --channel latest/edge --config default-gateway=kubeflow-gateway --trust
juju relate istio-pilot istio-ingressgateway

juju deploy kubeflow-roles --channel 1.8/stable --trust
juju deploy kubeflow-profiles --channel latest/edge --trust
juju deploy kubeflow-dashboard --channel latest/edge --trust
juju relate kubeflow-dashboard kubeflow-profiles
juju deploy admission-webhook --trust --channel 1.8/stable
juju relate istio-pilot:ingress kubeflow-dashboard:ingress
juju relate istio-pilot kubeflow-dashboard

juju deploy jupyter-ui --channel latest/edge --trust
juju deploy jupyter-controller --channel latest/edge --trust
juju relate istio-pilot jupyter-ui
juju relate kubeflow-dashboard:links jupyter-ui:dashboard-links

juju deploy katib-controller --channel 0.16/stable
juju deploy katib-db-manager --channel 0.16/stable --trust
juju deploy katib-ui --channel 0.16/stable --trust
juju deploy mysql-k8s --channel 8.0/stable --trust --constraints mem=2G katib-db
juju relate katib-db-manager:relational-db katib-db:database
juju relate istio-pilot katib-ui
juju relate kubeflow-dashboard:links katib-ui:dashboard-links

From that, I tested two ways: 1) By creating a notebook server, cloning this charmed-kubeflow-uats repo, and running through the notebook tests/notebooks/katib/katib-integration.ipynb manually 2) From my local machine, cloning this repo and running tox -e uats -- --filter "katib"

Results from (2) ``` tox -e uats -- --filter "katib" [11:07:40] uats: install_deps> python -I -m pip install -r requirements.txt uats: commands[0]> pytest -vv --tb native /home/scribs/code/canonical/charmed-kubeflow-uats/KF-5395-update-katib-test/driver/ -s --model kubeflow --filter katib ==================================================== test session starts ==================================================== platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0 -- /home/scribs/code/canonical/charmed-kubeflow-uats/KF-5395-update-katib-test/.tox/uats/bin/python cachedir: .tox/uats/.pytest_cache rootdir: /home/scribs/code/canonical/charmed-kubeflow-uats/KF-5395-update-katib-test configfile: pyproject.toml plugins: asyncio-0.21.1, anyio-4.0.0, operator-0.31.0 asyncio: mode=strict collected 2 items driver/test_kubeflow_workloads.py::test_create_profile ------------------------------------------------------ live log setup ------------------------------------------------------- INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/apiextensions.k8s.io/v1/customresourcedefinitions "HTTP/1.1 200 OK" INFO test_kubeflow_workloads:test_kubeflow_workloads.py:59 Creating Profile test-kubeflow... INFO httpx:_client.py:1013 HTTP Request: POST https://192.168.0.145:16443/apis/kubeflow.org/v1/profiles "HTTP/1.1 201 Created" ------------------------------------------------------- live log call ------------------------------------------------------- INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/kubeflow.org/v1/profiles/test-kubeflow "HTTP/1.1 200 OK" INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/api/v1/namespaces/test-kubeflow "HTTP/1.1 404 Not Found" INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/api/v1/namespaces/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:32 Waiting for namespace test-kubeflow to become 'Active': phase == Active PASSED driver/test_kubeflow_workloads.py::test_kubeflow_workloads ------------------------------------------------------- live log call ------------------------------------------------------- INFO test_kubeflow_workloads:test_kubeflow_workloads.py:100 Starting Kubernetes Job test-kubeflow/test-kubeflow to run notebook tests... INFO httpx:_client.py:1013 HTTP Request: POST https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs "HTTP/1.1 201 Created" INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == not ready) INFO utils:utils.py:40 Retrying in 2 seconds (attempts: 1) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == not ready) INFO utils:utils.py:40 Retrying in 4 seconds (attempts: 2) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == not ready) INFO utils:utils.py:40 Retrying in 8 seconds (attempts: 3) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == active) INFO utils:utils.py:40 Retrying in 16 seconds (attempts: 4) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == active) INFO utils:utils.py:40 Retrying in 32 seconds (attempts: 5) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == active) INFO utils:utils.py:40 Retrying in 32 seconds (attempts: 6) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == active) INFO utils:utils.py:40 Retrying in 32 seconds (attempts: 7) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == active) INFO utils:utils.py:40 Retrying in 32 seconds (attempts: 8) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == active) INFO utils:utils.py:40 Retrying in 32 seconds (attempts: 9) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:76 Waiting for Job test-kubeflow/test-kubeflow to complete (status == active) INFO utils:utils.py:40 Retrying in 32 seconds (attempts: 10) INFO httpx:_client.py:1013 HTTP Request: GET https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" INFO utils:utils.py:69 Job test-kubeflow/test-kubeflow completed successfully! INFO test_kubeflow_workloads:test_kubeflow_workloads.py:123 Fetching Job logs... ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-8.1.1, pluggy-1.4.0 -- /opt/conda/bin/python3.8 cachedir: .pytest_cache rootdir: /tests configfile: pytest.ini plugins: anyio-3.6.2 collecting ... collected 9 items / 8 deselected / 1 selected test_notebooks.py::test_notebook[katib-integration] -------------------------------- live log call --------------------------------- INFO test_notebooks:test_notebooks.py:44 Running katib-integration.ipynb... PASSED [100%] ================= 1 passed, 8 deselected in 137.94s (0:02:17) ================== PASSED ----------------------------------------------------- live log teardown ----------------------------------------------------- INFO test_kubeflow_workloads:test_kubeflow_workloads.py:72 Deleting Profile test-kubeflow... INFO httpx:_client.py:1013 HTTP Request: DELETE https://192.168.0.145:16443/apis/kubeflow.org/v1/profiles/test-kubeflow "HTTP/1.1 200 OK" INFO test_kubeflow_workloads:test_kubeflow_workloads.py:129 Deleting Job test-kubeflow/test-kubeflow... INFO httpx:_client.py:1013 HTTP Request: DELETE https://192.168.0.145:16443/apis/batch/v1/namespaces/test-kubeflow/jobs/test-kubeflow "HTTP/1.1 200 OK" =============================================== 2 passed in 224.94s (0:03:44) =============================================== uats: OK (242.85=setup[16.95]+cmd[225.89] seconds) congratulations :) (242.95 seconds) ```
ca-scribner commented 3 months ago

@NohaIhab are you sure? I've used the same katib example, upstream has just updated it to use a new container

NohaIhab commented 3 months ago

@ca-scribner you're right, I misunderstood the change in upstream