Closed peterzhuamazon closed 10 months ago
@peterzhuamazon, @bbarani completing this by 11/1 is a short notice with all the ongoing work. What OpenSearch release are we targeting this for?
@peterzhuamazon our CI workflow doesn't use the container
/image
keywords at all. Is there a standard we need to update to? We have a JENKINS_URL
env var set.
@peterzhuamazon our CI workflow doesn't use the
container
/image
keywords at all. Is there a standard we need to update to? We have aJENKINS_URL
env var set.
I will work with you on that soon. Github actions itself is using containers anyway. It is very easy to onboard a new container image and run the same.
We have a new approach established here. Please see this sample workflow file from Alerting for example: https://github.com/opensearch-project/alerting/blob/main/.github/workflows/multi-node-test-workflow.yml
Thanks.
@peterzhuamazon I'm still having troubles using that alerting change as reference. I've copied the Get-CI-Image-Tag, copied the job container configuration, and copied the shell cmds, but I'm still getting GLIBC errors. Ref: https://github.com/opensearch-project/anomaly-detection/actions/runs/6568513863/job/17843044510?pr=1080
Hi All,
This is coming from the campaign here:
Overview
We would like your CI check (specifically plugin build) in GitHub Repo to run on top of the Build Docker Images from production distribution pipeline.
This is to ensure every plugin repo will use the exact docker images we used in Jenkins build, to check their PRs and run tests before merging the code, so that issues can be detected earlier, and environment can be identical across teams.
Solutions
The Build Team has created a simple script to dynamically retrieve the current docker image name/tag, so everyone can easily pull the images for their CI checks.
We have a trial run of the above with k-NN team. The script retrieves the docker image dynamically, save output, and use it as the docker image to pull for the upcoming run:
Note that GitHub Actions only support LINUX docker container at the time of this writing, so we will add Windows containers later on as well as macOS.
Implementation Notes
We would like you to review above PR and implement similar changes. Note on line
33
of the above k-NN PR,-u
and-p
parameters needs to assign values accordingly.OpenSearch Plugin:
OpenSearch-Dashboards Plugin:
Note that in the above k-NN PR, despite it being OpenSearch plugin, it still uses
rockylinux8
, as we initially plan to upgrade to rockylinux. We have since revert back tocentos7
to support older versions of systems running k-NN lib. As a result, all OpenSearch plugins still usescentos7
for the time being, and all OpenSearch-Dashboards plugins can go torockylinux8
.Completion Date
The above should be implemented by
Nov. 1, 2023 (2023-11-01)
by Plugin Owners to their repository. And backport the changes to2.x
branch after merging inmain
branch.Contacts
Please contact @peterzhuamazon for any questions on this campaign.
cc: @bbarani
Thanks.