Publish docker images to docker hub or github packages

NOAA-OWP / wres

Code and scripts for the Water Resources Evaluation Service

Other

2 stars 1 forks source link

Publish docker images to docker hub or github packages #28

Closed epag closed 1 week ago

epag commented 2 weeks ago

Build and publish docker images used in a public registry as part of the deployment pipeline

epag commented 2 weeks ago

Hey @HankHerr-NOAA ,

Just want confirm there should be no issue posting our images to an external registry? I know current day we use an internal registry, but posting these to a public one will only further make it easier for external users to deploy their own service. This will help automate our deployment process as well. Let me know if you have any concerns

HankHerr-NOAA commented 2 weeks ago

If you use GitHub packages, then I think we should be safe since its all in GitHub. If you use something else, I'm not sure. If you want to use something other than GitHub Packages, let me know, and I'll ask around.

Note that, when we deploy, we'll need to pull from the NWCAL registry, since it was approved years ago for that purpose and includes image scanning. That is, unless we are able to use something like Iron Bank or another government-approved image registry. That's probably allowed for deployment.

Thanks,

Hank

epag commented 2 weeks ago

Do you know what we use to scan images/if we could do that in github if we could deploy directly from github packages?

HankHerr-NOAA commented 2 weeks ago

I believe the registry uses Claire/Clare/whatever.

I don't know if ITSG would sign off on deploying using images pulled from GitHub Packages. I would need to ask, and I'm not sure if they would even have an answer. If that is a strong enough preference, let me know, and I'll send an email.

Thanks,

hank

epag commented 2 weeks ago

I need to do some investigation, but I believe our flow could be as follows if we are allowed to do this:

Mark a release as pre-release
The CI/CD will be kicked off that builds/test the code
The images are then created and checked for vulnerabilities before being uploaded to a public registry
We download the compose-entry/worker onto staging directly from github and deploy (No need to build/push images anymore)

I think that this would be the most stream lined that we would be able to get in terms of a CI/CD pipeline. I found a github action for a clair scan, so it could be possible if ITSG is okay with it

epag commented 1 week ago

Unlikely we will be able to do this

epag commented 1 week ago

Thinking more about this, I don't think it makes much sense to merge the steps that create and commit the compose entry and worker files into the pipeline.

Since we will still need to push the images on the gov hardware we will still need to run a script and also since we have branch protections enabled a user still needs to be involved in commit these changes even if they were auto generated. Going to close this and I think we are done with automation until we reduce/store our test data somewhere