Update of Habana SynapseAI notebook to 1.15.1

Xaenalt commented 4 months ago

Description

Adds the SynapseAI 1.15.1 notebook, which is needed for RHOAI 2.10 compatibility

This required quite a few changes, since the SynapseAI stack requires Python 3.11 for 1.15.1. Uses a similar workflow to Anaconda with a custom base image, this however should be easy to update in the future just by swapping out the FROM in the base image and verifying the python versions going forward.

How Has This Been Tested?

Builds successfully, could use some functional testing

Merge criteria:

[X] The commits are squashed in a cohesive manner and have meaningful messages.
[X] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[X] The developer has manually tested the changes and verified that the changes work

Xaenalt commented 4 months ago

@harshad16 PTAL

Xaenalt commented 4 months ago

/retest

harshad16 commented 4 months ago

@Xaenalt , i m not sure if we would like to maintain python 3.10 to be honest we should have this checked with others. i understand this version of Habana only is available at 3.10, however without confirmation this would make it more harder for the team to maintain this in long run.

harshad16 commented 4 months ago

/hold

Xaenalt commented 4 months ago

Yeah, thankfully the same versions of everything exist in 3.10, but yeah let's have a longer discussion about it

Xaenalt commented 4 months ago

According to Intel, this would only be necessary until the 1.17 release which will be end of July-ish

openshift-ci[bot] commented 4 months ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign harshad16 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/opendatahub-io/notebooks/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

jiridanek commented 4 months ago

@Xaenalt , i m not sure if we would like to maintain python 3.10 to be honest

Red Hat ships Python 3.11, can you wait and use that? It is supported until its retirement date in May 2026 in rhel 8 as well as rhel 9. If you base the image on ub9/rhel9 and use Python 3.12 available there, that has retirement date in Apr 2027. https://access.redhat.com/support/policy/updates/rhel-app-streams-life-cycle#rhel8_application_streams

Xaenalt commented 4 months ago

If we're good to use Python 3.11 that'll be excellent, and will be supported in the 1.17 release

I was just doing a few small fixes to the notebook PR, since it's a much easier to use template for the future

Xaenalt commented 4 months ago

If possible, I'd like to revisit the timeline until 1.17 (EO July timeframe) at which point we can retire the 1.15/1.16 series which are stuck on Python 3.10. Currently (pre 1.17) the notebook SynapseAI version has to match what's in the operator

jstourac commented 2 months ago

@Xaenalt JFYI, I updated the description of the https://issues.redhat.com/browse/RHOAIENG-5404 which is tracking addition of the Gaudi v1.17 into RHOAI as v1.17 seems to be out now.

In the meantime, there is a plan to introduce a Python 3.11 via #659 (just the image definitions but won't be used in the actual RHOAI/ODH builds yet - that will be a next step).

openshift-ci[bot] commented 1 month ago

@Xaenalt: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/runtime-intel-pyt-ubi9-python-3-9-pr-image-mirror	06cfbcd1639a784b75acf13567d275e97aa0750c	link	true	`/test runtime-intel-pyt-ubi9-python-3-9-pr-image-mirror`
ci/prow/notebook-jupyter-intel-pyt-ubi9-python-3-9-pr-image-mirror	06cfbcd1639a784b75acf13567d275e97aa0750c	link	true	`/test notebook-jupyter-intel-pyt-ubi9-python-3-9-pr-image-mirror`
ci/prow/notebook-jupyter-intel-tf-ubi9-python-3-9-pr-image-mirror	06cfbcd1639a784b75acf13567d275e97aa0750c	link	true	`/test notebook-jupyter-intel-tf-ubi9-python-3-9-pr-image-mirror`
ci/prow/notebooks-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test notebooks-e2e-tests`
ci/prow/images	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test images`
ci/prow/habana-notebooks-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test habana-notebooks-e2e-tests`
ci/prow/anaconda-ubi8-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test anaconda-ubi8-e2e-tests`
ci/prow/amd-runtimes-ubi9-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test amd-runtimes-ubi9-e2e-tests`
ci/prow/notebook-rocm-ubi9-python-3-9-pr-image-mirror	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test notebook-rocm-ubi9-python-3-9-pr-image-mirror`
ci/prow/runtime-rocm-pytorch-ubi9-python-3-9-pr-image-mirror	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test runtime-rocm-pytorch-ubi9-python-3-9-pr-image-mirror`
ci/prow/runtime-rocm-tensorflow-ubi9-python-3-9-pr-image-mirror	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test runtime-rocm-tensorflow-ubi9-python-3-9-pr-image-mirror`
ci/prow/runtimes-ubi8-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test runtimes-ubi8-e2e-tests`
ci/prow/notebook-rocm-jupyter-tf-ubi9-python-3-11-pr-image-mirror	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test notebook-rocm-jupyter-tf-ubi9-python-3-11-pr-image-mirror`
ci/prow/notebook-cuda-jupyter-tf-ubi9-python-3-11-pr-image-mirror	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test notebook-cuda-jupyter-tf-ubi9-python-3-11-pr-image-mirror`
ci/prow/notebook-jupyter-pytorch-ubi9-python-3-11-pr-image-mirror	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test notebook-jupyter-pytorch-ubi9-python-3-11-pr-image-mirror`
ci/prow/runtimes-ubi9-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test runtimes-ubi9-e2e-tests`
ci/prow/rocm-runtimes-ubi9-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test rocm-runtimes-ubi9-e2e-tests`
ci/prow/notebooks-ubi9-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test notebooks-ubi9-e2e-tests`
ci/prow/codeserver-notebook-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test codeserver-notebook-e2e-tests`
ci/prow/intel-notebooks-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test intel-notebooks-e2e-tests`
ci/prow/rstudio-notebook-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test rstudio-notebook-e2e-tests`
ci/prow/rocm-notebooks-e2e-tests	4009767fee480335aebdda627e797f7b43c833dd	link	true	`/test rocm-notebooks-e2e-tests`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).

jstourac commented 1 month ago

I think that this work will be superseded by the #695.

Xaenalt commented 1 month ago

Yep, superseded by #695

opendatahub-io / notebooks