search

KwanLab / Autometa

Autometa: Automated Extraction of Genomes from Shotgun Metagenomes
https://autometa.readthedocs.io
Other
40 stars 15 forks source link

:fire::whale::green_heart: Fix docker-builds by removing pinned dependencies #340

Closed evanroyrees closed 1 year ago

evanroyrees commented 1 year ago
chasemc commented 1 year ago

Add schedule: cron: '0 0 *' (i.e. midnight) build for Autometa docker images to catch when builds are failing

This would work if there's a network error or similar but seems like a hacky workaround to whatever the problem is.

evanroyrees commented 1 year ago

This would work if there's a network error or similar but seems like a hacky workaround to whatever the problem is.

I'm not sure what you mean by this?

Just to make sure we are on the same page, the following is why I've included a scheduled docker image build as a github action:

schedule: 
    - cron: '0 0 * * *'  # every day at midnight

TL;DR Pinning dependencies: good for reproducibility, bad for maintenance

Best of both worlds? 🌎 🤷‍♂️ --> remove any unnecessary pins, upload artifact of explicit versions after successful env creation

The addition of the schedule to the Github actions (shown above) is to make sure the build process is keeping up to date with library dependencies. Previously the docker image build action would only be triggered on a release or PR, thus not tracking the most up to date versions of any library dependencies available. The issue is more likely to arise when a release has not occurred for long periods of time (in this case ~8 months). In this scenario, library dependencies have continued to mature and may cause (was causing) the current build to fail. This is (and was) not apparent however when checking Dockerhub as the image was successfully constructed and is currently available (albeit from 8 months ago). This has led to confusion for developers as the image was successfully constructed, but is no longer building due to library dependencies major/minor version releases and patches. Of course pinning all dependencies to the old versions may resolve this, but now we are stuck discarding all of the effort developers have put in to reach these version bumps and only have deprecated packages to gain.

A scheduled job would alert the maintainers as soon as the build fails allowing maintainers to catch when the builds start failing in a relatively short period of time (i.e. a day instead of 8 months). I do not see a more appropriate work-around other than to re-build the docker image any time any of the dependencies bump a version (Maybe this is a github action that is currently available that I'm not aware of?). An additional step in the docker image build action may provide the best of both worlds, i.e. these being explicitly pinned dependencies and the most up-to-date versions. This step would export dependency versions after successful image creation, perhaps via an artifact upload or something of the like while also not having the pins defined in the actual repo autometa-env.yaml so as to avoid missing any important updates that others have implemented for the various libraries and tools used by Autometa.

chasemc commented 1 year ago

That conflates all dev work with ever-changing dependency management (ie I could make a PR that is valid today but fails tomorrow if a dependency changes).

I do not see a more appropriate work-around other than to re-build the docker image any time any of the dependencies bump a version (Maybe this is a github action that is currently available that I'm not aware of?).

This is pretty common practice, e.g. GH has dependabot that can watch requirements.txt etc and make a PR when dependencies change.

chasemc commented 1 year ago

https://docs.github.com/en/code-security/dependabot/dependabot-version-updates

evanroyrees commented 1 year ago

I was not able to get docker to build an image from micromamba using version 1.1.0 as the base image. That is not causing any issues at the moment, so perhaps for another day.

As it stands now if the nightly build fails, a notification (based on user settings/preferences) will be sent. Otherwise this will not change any of the other images (e.g. main, dev, release-*)