[release]: Opensearch Migration ECR Images

[AndreKurait]

Did you read the on-boarding document

Yes

What is the name of your component?

Opensearch Migrations

What is the link to your GitHub repo?

https://github.com/opensearch-project/opensearch-migrations/

Targeted release date

07/26/24 in order to unblock existing project workstreams

Where should we publish this component?

ECR -

• opensearchproject/opensearch-migrations/traffic-replayer
• opensearchproject/opensearch-migrations/capture-proxy
• opensearchproject/opensearch-migrations/migration-console
• opensearchproject/opensearch-migrations/reindex-from-snapshot

What type of artifact(s) will be generated for this component?

OCI Images

Have you completed the required reviews including security reviews, UX reviews?

Existing security reviews covering the solution code. No UX review needed as it is a command line solution.

We're looking to get the scaffolding set up so that we are generating the docker images in a format that can be published to the public ECR so that we can include them in our current security review.

Have you on-boarded automated security scanning for the GitHub repo associated with this component?

Yes

Additional context

This is an extension of https://github.com/opensearch-project/opensearch-build/issues/4592

[Divyaasm]

Hi, @AndreKurait Can you please mention where you currently store the docker images. And do you build and maintain the images on dockerhub or ecr right now?

[peterzhuamazon]

To add a bit more context, our team will not build the image for opensearch-migrations since it is not our product. We will copy the image you built and release on dockerhub/ecr prod repos.

You would need to have automation steps on your side to build the images before we pick up.

Thanks.

[peterzhuamazon]

Hi @AndreKurait would you mind give an update to above questions?

Thanks.

[AndreKurait]

We plan to build the images using github actions. Original communication with @Divyaasm resulted in a recommendation to have our github action publish to dockerhub staging then jenkins copy to dockerhub and ecr prod. Similar to opensearch-benchmark https://github.com/opensearch-project/opensearch-benchmark/blob/main/.github/workflows/docker.yml

[AndreKurait]

Recent communication with @peterzhuamazon included a change in guidance to instead follow data-prepper's process to publish to ECR first.

We can follow this process instead. @peterzhuamazon, can you provide guidance on the account and ECR to use for this? Do you want this to be a shared account or one we control? Should this be a private or public repo?

[peterzhuamazon]

Hi @AndreKurait ,

Approach 1: GitHub to DockerHub

• To make sure secret is safely guarded, we will setup configure-aws-credentials on your action
• The action will retrieve the secret from our secret manager, and you can use it to publish to dockerhub
• In order to publish to production, a jenkins workflow will needs to be created, with webhook connected to your repo, so that release tag cut can trigger a publishing to prod

Approach 2: ECR to DockerHub

• Your team manage your own AWS account and ECR
• You manage your workflow or github actions to push to ECR
• Our workflow will copy your image directly from ECR and push to dockerhub staging, then publish to prod, same jenkins workflow + webhook approach as the above

Please let us know which approach your team would like to try. The reason I recommend ECR because your initial ask is ECR, and you can choose based on your need.

Thanks.

[AndreKurait]

Hi @peterzhuamazon, We'd like to follow Approach 1.

I've created GHA steps in https://github.com/opensearch-project/opensearch-migrations/pull/847 included commented out code for how you would publish after logging in.

Could you please configure the secrets and provide a PR for that action?

[peterzhuamazon]

Offline discussion with @AndreKurait and we will setup secret manager entries for them regarding dockerhub staging credentials.

Will make some changes before updating their repo workflow.

Thanks.

[peterzhuamazon]

Test in beta env with the secrets entries success. Will send a PR to the migrations repo soon.

Thanks.