search

bl1231 / bilbomd-worker

Processes BilboMD jobs and run CHARMM, FoXS, and MultiFoXS
1 stars 0 forks source link

Enhance Workflow for Building Docker Images Using GitHub Actions #337

Closed shrprabh closed 5 months ago

shrprabh commented 5 months ago

We need to enhance our current workflow by implementing a GitHub Actions pipeline that automates the creation of Docker images dynamically. This enhancement aims to streamline the image build process, ensuring that every push or pull request triggers an automated build, test, and deployment sequence. The new workflow should support the following features:

Automated Builds: Automatically build Docker images on push and pull requests to specified branches.

Dynamic Tagging: Tag images with commit hashes, branch names, and version numbers.

Testing: Integrate a testing step to ensure the built image functions correctly.

Multi-Platform Builds: Support multi-platform builds to create images for different architectures (e.g., x86, ARM).

Publishing: Automatically push the built images to our Github registry.

shrprabh commented 5 months ago

Enhancement: Automated Docker Image Build for bilbomd-spin-worker

In the bilbomd-worker project, there are three Dockerfiles serving different deployment needs:

  1. bilbomd-worker.dockerfile: Builds an image specifically for deployment at ALS, which is not required for NERSC/SPIN environments.

  2. bilbomd-perlmutter-worker.dockerfile: Constructs an image tailored for executing the BilboMD pipeline on compute nodes. Automation for this Dockerfile build is deemed unnecessary.

  3. bilbomd-spin-worker.dockerfile: This Dockerfile generates an image used for SPIN, devoid of dependencies on CHARMM or OpenMM source code. It is proposed that this Dockerfile be automated using GitHub Actions for streamlined build processes.

The proposal suggests focusing automation efforts exclusively on the bilbomd-spin-worker.dockerfile to enhance efficiency and maintain consistency across SPIN deployments.

The Build supports ARM64 and AMD64 architecture

dsclassen commented 5 months ago

I have tested the ability to pull and deploy the docker images for bilbomd-spin-worker from ghcr.io on NERSC/SPIN and it seems to work. Perlmutter is in maintenance mode today so I can't fully test things, but I think it will probably work.

shrprabh commented 5 months ago

Hi @dsclassen , Our current CI action will tag images with latest and a date+githash: tags: | ghcr.io/${{ github.repository }}/bilbomd-spin-worker:latest ghcr.io/${{ github.repository }}/bilbomd-spin-worker:${{ steps.version.outputs.VERSION }} This seems good. So latest will always be the latest successfully built image, but if latest breaks something, we can always point to a specific date+githash tag if needed.

I have applied this logic to bilbomd-backend. I am currently checking how much time it will take to build. After that, I will apply the same logic to bilbomd-ui. Once this is done, we can perform a sanity check. Later, I will change the branch name while raising the PR.