search

opendatahub-io / notebooks

Notebook images for ODH
Apache License 2.0
15 stars 51 forks source link

[GHA] digest updater workflow fixes and updates #566

Closed jstourac closed 2 weeks ago

jstourac commented 3 weeks ago

This contains changes for the digest update workflow for the update of notebook images and commit shas. There are couple of fixes and a small refactoring done, more details in particular commits:

Truth is that I don't fully understand, how is possible that in this case #565 the rstudio images were selected properly :thinking:

https://issues.redhat.com/browse/RHOAIENG-8813

How I tested, check my comment below. As stated there, my plan was also to deduplicate the code, but it's WIP on my machine right now, so if this is needed, we can merge this and I will raise another followup PR after my PTO.

How Has This Been Tested?

Right now, it hasn't been tested, will do later.

Merge criteria:

openshift-ci[bot] commented 3 weeks ago

Skipping CI for Draft Pull Request. If you want CI signal for your change, please convert it to an actual PR. You can still manually trigger a test run with /test all

jstourac commented 3 weeks ago

~Current implementation working example:~ ~ https://github.com/opendatahub-io/notebooks/compare/main...jstourac:notebooks:digest-updater-9584749053~ ~ https://github.com/jstourac/notebooks/actions/runs/9584749053~

~Looks good except that I don't understand why it skipped the habana image update~ :thinking:

~Execution time was 33minutes vs 45minutes in upstream.~

Okay, mystery solved - reason was that I ran it from the main branch instead of the my branch with the fixes :facepalm: so ignore the above.

jstourac commented 3 weeks ago

Ah, unfortunately GH uses Ubuntu 22.04 LTS, which has Skopeo in old version and doesn't contain the --no-tags option: https://manpages.ubuntu.com/manpages/jammy/man1/skopeo-inspect.1.html. So we have to live without it until a newer OS is introduced (a little bit slower for now).

jstourac commented 3 weeks ago

Anyway, this looks good:

So, unless our check job is faulty, the changes in this PR should be fine.

Didn't check the security analysis, though - but we plan to review and update it anyway, so probably not a big deal right now.

Also, code deduplication using GH job matrix is WIP on my machine locally right now. But looks like current content is ready for review and comments - so if it is required, I can open a followup PR for the code deduplication later.

openshift-ci[bot] commented 2 weeks ago

@jstourac: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/notebooks-e2e-tests 0c6aecc820f5323a3b39243464beb9c06fd151a5 link true /test notebooks-e2e-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
jiridanek commented 2 weeks ago

/test notebooks-ubi9-e2e-tests

atheo89 commented 2 weeks ago

Thanks Jan for the update! It looks great now!

I tested this locally and works as expected. image

On downstream, we will need to open a separate PR to apply the same fixes. In short, on downstream we fetch images from different registries for almost every notebook, also the name of the workflow is different from the one on upstream.

/lgtm /approve /override ci/prow/intel-notebooks-e2e-tests

openshift-ci[bot] commented 2 weeks ago

@atheo89: Overrode contexts on behalf of atheo89: ci/prow/intel-notebooks-e2e-tests

In response to [this](https://github.com/opendatahub-io/notebooks/pull/566#issuecomment-2191456506): >Thanks Jan for the update! It looks great now! > >I tested this locally and works as expected. >![image](https://github.com/opendatahub-io/notebooks/assets/42587738/70e208bc-3d11-40e4-bc15-035580ebe154) > >On downstream, we will need to open a separate PR to apply the same fixes. In short, on downstream we fetch images from different registries for almost every notebook, also the [name](https://github.com/red-hat-data-services/notebooks/blob/main/.github/workflows/notebook-digest-updater.yaml) of the workflow is different from the one on upstream. > >/lgtm >/approve >/override ci/prow/intel-notebooks-e2e-tests Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
openshift-ci[bot] commented 2 weeks ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: atheo89, jiridanek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/opendatahub-io/notebooks/blob/main/OWNERS)~~ [atheo89] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
jiridanek commented 2 weeks ago

/cherrypick 2024a /cherrypick 2023b /cherrypick 2023a

openshift-cherrypick-robot commented 2 weeks ago

@jiridanek: once the present PR merges, I will cherry-pick it on top of 2024a in a new PR and assign it to you.

In response to [this](https://github.com/opendatahub-io/notebooks/pull/566#issuecomment-2191485560): >/cherrypick 2024a >/cherrypick 2023b >/cherrypick 2023a Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
jiridanek commented 2 weeks ago

/cherrypick 2023b

openshift-cherrypick-robot commented 2 weeks ago

@jiridanek: once the present PR merges, I will cherry-pick it on top of 2023b in a new PR and assign it to you.

In response to [this](https://github.com/opendatahub-io/notebooks/pull/566#issuecomment-2191486134): >/cherrypick 2023b Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
jiridanek commented 2 weeks ago

/cherrypick 2023a

openshift-cherrypick-robot commented 2 weeks ago

@jiridanek: once the present PR merges, I will cherry-pick it on top of 2023a in a new PR and assign it to you.

In response to [this](https://github.com/opendatahub-io/notebooks/pull/566#issuecomment-2191486623): >/cherrypick 2023a Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
openshift-cherrypick-robot commented 2 weeks ago

@jiridanek: new pull request created: #592

In response to [this](https://github.com/opendatahub-io/notebooks/pull/566#issuecomment-2191485560): >/cherrypick 2024a >/cherrypick 2023b >/cherrypick 2023a Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
openshift-cherrypick-robot commented 2 weeks ago

@jiridanek: #566 failed to apply on top of branch "2023b":

Applying: fix the regexp used for the latest image tag selection
Using index info to reconstruct a base tree...
M   .github/workflows/notebooks-digest-updater-upstream.yaml
A   .github/workflows/runtimes-digest-updater-upstream.yaml
A   ci/security-scan/quay_security_analysis.py
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): ci/security-scan/quay_security_analysis.py deleted in HEAD and modified in fix the regexp used for the latest image tag selection. Version fix the regexp used for the latest image tag selection of ci/security-scan/quay_security_analysis.py left in tree.
CONFLICT (modify/delete): .github/workflows/runtimes-digest-updater-upstream.yaml deleted in HEAD and modified in fix the regexp used for the latest image tag selection. Version fix the regexp used for the latest image tag selection of .github/workflows/runtimes-digest-updater-upstream.yaml left in tree.
Auto-merging .github/workflows/notebooks-digest-updater-upstream.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 fix the regexp used for the latest image tag selection
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
In response to [this](https://github.com/opendatahub-io/notebooks/pull/566#issuecomment-2191486134): >/cherrypick 2023b Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
openshift-cherrypick-robot commented 2 weeks ago

@jiridanek: #566 failed to apply on top of branch "2023a":

Applying: fix the regexp used for the latest image tag selection
Using index info to reconstruct a base tree...
A   .github/workflows/notebooks-digest-updater-upstream.yaml
A   .github/workflows/runtimes-digest-updater-upstream.yaml
A   ci/security-scan/quay_security_analysis.py
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): ci/security-scan/quay_security_analysis.py deleted in HEAD and modified in fix the regexp used for the latest image tag selection. Version fix the regexp used for the latest image tag selection of ci/security-scan/quay_security_analysis.py left in tree.
CONFLICT (modify/delete): .github/workflows/runtimes-digest-updater-upstream.yaml deleted in HEAD and modified in fix the regexp used for the latest image tag selection. Version fix the regexp used for the latest image tag selection of .github/workflows/runtimes-digest-updater-upstream.yaml left in tree.
CONFLICT (modify/delete): .github/workflows/notebooks-digest-updater-upstream.yaml deleted in HEAD and modified in fix the regexp used for the latest image tag selection. Version fix the regexp used for the latest image tag selection of .github/workflows/notebooks-digest-updater-upstream.yaml left in tree.
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 fix the regexp used for the latest image tag selection
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
In response to [this](https://github.com/opendatahub-io/notebooks/pull/566#issuecomment-2191486623): >/cherrypick 2023a Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
harshad16 commented 2 weeks ago

@jiridanek , please don't cherrypick pr in 2024a , as we would sync them with teh new automation. thanks