search

psychoinformatics-de / inm-icf-utilities

MIT License
0 stars 5 forks source link

Pre-built container expired (link in docs is broken) #49

Closed mslw closed 2 months ago

mslw commented 3 months ago

The Containerized execution docs page (docs/source/reference/container.rst) contains a link to the pre-built container, which points to Appveyor's artifacts directory. This link is currently broken, returning only:

{"message":"Artifact not found or access denied."}

Appveyor's retention policy is to remove artifacts older than 3 months (for paid accounts). Now that the tooling is in maintenance mode, rather than active development, relying on CI is no longer sufficient to provide a downloadable image.

We can go for another deployment method, e.g. it would be easy to set up a git-annex repository with the image "releases" and host them elsewhere - the downside is that the release process would need to be manual unless we want to store some secrets with GitHub.

From user's POV, the container image can also be easily built with singularity build --fakeroot icf.sif singularity/icf.def (or the same with apptainer) from this repository checkout. The container size is currently 222 MB.

Reported by: Stefanie

adswa commented 2 months ago

https://ci.appveyor.com/project/mih/datalad-ci-docker-containers is the appveyor project, I believe. I think hosting this more persistently sounds useful to me, though.

mslw commented 2 months ago

Also to check: can we run periodic jobs on appveyor?

christian-monch commented 2 months ago

How about the following: since the recipes and container creation scripts are contained in the github project https://github.com/datalad/datalad-ci-docker-containers, we could release the project and add the artifacts to the release. AFAIK, those should be available "forever".

Github also supports to link to the latest asset with a specific name. That means, the documentation can contain a static link to the latest released container images

mslw commented 2 months ago

https://ci.appveyor.com/project/mih/datalad-ci-docker-containers is the appveyor project, I believe. I think hosting this more persistently sounds useful to me, though.

No, the ICF container is built in this repository's appveyor project https://ci.appveyor.com/project/mih/inm-icf-utilities (the download link is https://ci.appveyor.com/api/projects/mih/inm-icf-utilities/artifacts/icf.sif).

After recent PRs, the artifact is available for download again, so the issue could be closed, but it will show up again in 3 months.

I found scheduled builds section in the appveyor docs, but it does not explain much. There is, however, a "build schedule" input box in project's general configuration - maybe it's enough to add a crontab schedule there, e.g. monthly (0 0 1 * *) or bimonthly (0 0 1 */2 *)?

mslw commented 2 months ago

I entered the "Build schedule": 20 3 1 */2 * (at 3:20am on the 1st of every 2nd month; thanks, https://cron.help) - I think the setting got saved - we'll see in September.

While moving to a more persistent storage still remains an option, this should close the issue. Thanks for the comments!