search

csiro-crop-informatics / repset

Reproducible evaluation of short read mappers
GNU General Public License v3.0
2 stars 3 forks source link

Singularity builds #20

Closed rsuchecki closed 5 years ago

rsuchecki commented 5 years ago

Singularity containers built from docker pulls may not be fully consistent

WARNING: pull for Docker Hub is not guaranteed to produce the
WARNING: same image on repeated pull

This may not be a huge issue since we'd rather run the final version of the pipeline on AWS batch which means using docker containers directly. Nevertheless it would be good to have fixed singularity containers on shub for each docker container on docker hub. Currently automation on shub does not appear to be flexible enough to integrate it with the procedure developed for docker hub. For example, you can only point automated builds to already existing GitHub branches 👎

rsuchecki commented 5 years ago

Points raise above still stand, but here is a proof-of-concept setup which includes Docker automated builds tagged with version+SHA and Singularity builds derived from these version-specific docker builds https://github.com/rsuchecki/miniconda3

rsuchecki commented 5 years ago

Final runs to be done on AWS batch so using Docker containers directly. Above mentioned solution to be discussed as a way of addressing the problem.