Add example trigger showing how to auto-trigger GitOps Update pipeline from MLOps build pipeline

grdryn commented 3 months ago

JIRA: https://issues.redhat.com/browse/RHOAIENG-4982

This commit is stacked upon #267, and will show the changes from that PR until it's merged. This is one approach that could be taken–I'll open a separate PR with an alternative approach.

Description

This example shows how users can choose to optionally trigger the GitOps Update pipeline at the end of a successful run of the MLOps build pipeline.

Advantages of this approach

It's optional
It doesn't require making the main build pipeline extra complex, by needing to know everything about the GitOps update pipeline (it doesn't need to know all of the parameters, or tasks; as other approaches might).
GitOps Update pipeline is still executable in different ways, and can ideally change without needing to also make changes in the build pipeline

Disadvantages of this approach

As it relies on an EventListener, an extra pod needs to be running all the time
Currently an additional trigger (and possibly additional EventListener, although interceptors might be able to ensure only the appropriate trigger(s) is/are fired on a single EventListener) is needed for each different model image and/or GitOps repo.

How Has This Been Tested?

[x] Ran the build pipeline without adding a non-empty value for the new gitops-update-pipeline-eventlistener-url param, and verified that the optional task to call the EventListener was skipped
[x] Ran the build pipeline with a valid webhook URL for the new gitops-update-pipeline-eventlistener-url param, and verified that the new optional task was executed, and the GitOps Update pipeline was triggered and completed successfully.

Merge criteria:

[x] The commits are squashed in a cohesive manner and have meaningful messages.
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has manually tested the changes and verified that the changes work

openshift-ci[bot] commented 3 months ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please ask for approval from grdryn. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/opendatahub-io/ai-edge/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

MarianMacik commented 2 months ago

Currently an additional trigger (and possibly additional EventListener, although interceptors might be able to ensure only the appropriate trigger(s) is/are fired on a single EventListener) is needed for each different model image and/or GitOps repo.

I don't think it is needed for each different model image because model image is directly in the image-registry-repo parameter, isn't it? Also, GitOps repository can be parameterized as well, right? In case a user has some internal organization for different GitOps repositories, they only need to change image registry, digest and GitOps repo URL. Or if everything is in one repository, then the yq-script must be changed accordingly.

grdryn commented 2 months ago

Currently an additional trigger (and possibly additional EventListener, although interceptors might be able to ensure only the appropriate trigger(s) is/are fired on a single EventListener) is needed for each different model image and/or GitOps repo.

I don't think it is needed for each different model image because model image is directly in the image-registry-repo parameter, isn't it? Also, GitOps repository can be parameterized as well, right? In case a user has some internal organization for different GitOps repositories, they only need to change image registry, digest and GitOps repo URL. Or if everything is in one repository, then the yq-script must be changed accordingly.

If anything other than the image-registry-repo and image-digest parameters need different values for different models and/or GitOps repos (and I expect that they typically will), then a different trigger should be used. I think that's a cleaner approach than having the (already quite cumbersome) aiedge-e2e pipeline needing to know all of the extra parameters for the GitOps Update pipeline.

MarianMacik commented 2 months ago

If anything other than the image-registry-repo and image-digest parameters need different values for different models and/or GitOps repos (and I expect that they typically will), then a different trigger should be used. I think that's a cleaner approach than having the (already quite cumbersome) aiedge-e2e pipeline needing to know all of the extra parameters for the GitOps Update pipeline.

I see, so these extra parameters would be hard-coded in the trigger template so the aiedge-e2e pipeline doesn't need to have these at the start.

opendatahub-io / ai-edge