docker-builds
This repository contains the Dockerfiles and other assorted files necessary for building Docker images for a variety of programs used by members of the StaPH-B (State Public Health Lab Bioinformatics) consortium. The purpose of this repository is to provide a centralized location for Docker images that is easily accessible for users, with clear documentation on how the containers were built and how to use them.
There are several community projects that create and share containers for bioinformatic tools. This repository contains the Dockerfiles and other assorted files necessary for building Docker images for a variety of tools used by members of the StaPH-B (State Public Health Lab Bioinformatics) consortium. The purpose of this repository is to provide a centralized location for Docker images that is easily accessible for users, with clear documentation on how the containers were built and how to use them.
This is a community resource, built and maintined by users from varied backgrounds and expertise levels. As such, we have provided some templates for contributing to this repository. If you would like to add a Docker image or improve upon the existing images, please fork the repository, make your changes/additions, and submit a pull request. If you are having an issue with an existing image, please submit an issue. We welcome any and all feedback!
See more details on how to contribute here
Docker image repositories & hosting
We host all of our docker images on two different repositories:
The development process of creating a new image is summarized as follows:
graph TD
A[fork staphb/docker-builds repo]-->B[create tool/version directory]
B-->C[create readme]
B-->D[create dockerfile]
D-->G[create app and test layers]
A-->E[add License to Program_Licenses.md]
A-->F[add tool to list in this readme]
E-->H[submit PR]
F-->H
G-->H
C-->H
H-->I{build to test}
I--success-->J[PR merged and docker image pushed to dockerhub and quay]
I--failure-->K[edit dockerfile]
K-->IUser Guide
The StaPH-B Docker User Guide was created to outline methods and best practices for using and developing docker containers. There are chapters for:
- Contributing
- Downloading Docker images
- Running Docker containers
- Developing Docker images
- Useful links
Summarized usage guide for docker
# Build a docker image to the 'test' layer
docker build --tag tool:test --target test <directory to Dockerfile>
docker build --tag samtools:test --target test samtools/1.15
# Download a docker image from dockerhub (most tools have a 'latest' version tag)
docker pull staphb/tool:version
docker pull staphb/shigatyper:2.0.2
# Run the container (don't forget to mount your volumes!)
docker run --rm -u $(id -u):$(id -g) -v <local directory>:/data tool:version <command>
docker run --rm -u $(id -u):$(id -g) -v $(pwd)/amrfinder_test_files:/data amrfinder:3.10 amrfinder --nucleotide 2021CK-01854_contigs.fa --threads 20 --name 2021CK-01854 --output /data/2021CK-01854.txt --organism KlebsiellaFurther documentation can be found at docs.docker.com
Templates
Several template files are provided. These are intended to be copied and edited by contributors.
- dockerfile-template/Dockerfile is the basic template useful for most images
- dockerfile-template/Dockerfile_mamba is a basic template for using the micromamba base image
- dockerfile-template/README.md is a basic readme file template to assist others in using the image
What about Singularity?
For many people, Docker is not an option, but Singularity is. Most Docker containers are compatible with Singularity and can easily be converted to Singularity format. Please see the User Guide for instructions on how to download docker images from dockerhub and how to run them using Singularity. We've worked hard to ensure that our containers are compatibile with Singularity, but if you find one that isn't, please leave an issue and let us know!
Summarized usage guide for singularity
# Pulling a container from dockerhub (creates a file)
singularity pull --name <name of singularity file> docker://staphb/bbtools:38.96
singularity pull --name staphb-bbtools-38.96.simg docker://staphb/bbtools:38.96
# Running the container (don't forget to mount your volumes!)
singularity exec --bind <local directory>:/data <name of singularity file> <command>
singularity exec --bind $(pwd)/fastq:/data staphb-bbtools-38.96.simg bbduk.sh in1=sample1_R1.fastq.gz in2=sample1_R2.fastq.gz out1=bbduk/sample1_rmphix_R1.fastq.gz out2=bbduk/sample1_rmphix_R2.fastq.gz outm=bbduk/sample1.matched_phix.fq ref=/opt/bbmap/resources/phix174_ill.ref.fa.gz stats=bbduk/sample1.phix.stats.txt threads=4Further documentation can be found at docs.sylabs.io
Training
- Training materials (slides & exercises) from the North East Bioinformatics Regional Resource 2023 workshop Docker for Public Health Bioinformatics can be located here, within
training/NE-BRR-docker-for-PH-bioinformatics-May2023/ - Video training from APHL 2024 workshop Intermediate Docker Bioinformatics Workshop can be located in APHL's e-learning resources
Logs
In November 2020, Docker began to implement pull rate limits for images hosted on dockerhub. This limits the number of docker pull's per time period (e.g. anonymous users allowed 100 pulls per six hours). We applied and were approved for Docker's "Open Source Program," which should have removed the pull rate limits for all staphb docker images! 🎉 🥳 If you encounter an error such as ERROR: toomanyrequests: Too Many Requests. or You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits. , please let us know by submitting an issue.
A huge thank you goes to the folks at Docker for supporting our efforts to distribute & share critical tools for public health bioinformatics. This has been especially important during the COVID-19 global pandemic, as many of these tools are used to conduct genomic surveillance on the SARS-CoV-2 virus as well as other important pathogens of public health concern.
To learn more about the docker pull rate limits and the open source software program, please see these blog posts (1, 2, and 3) and Docker documentation (1).
NOTE: In the table below, we do not provide individual links to the various tools on quay.io, please visit the above quay.io link to find all of our docker images.
Available Docker images
You can also view the list of images on Docker hub here: https://hub.docker.com/r/staphb/
License
- GNU GPLv3 license was added 2020-01-16
- We keep a list of the licenses for the main software within the docker images here: Program_Licenses.md
- Links to licenses for each program should also be listed as a metadata
LABELwithin each dockerfile
Authors/Maintainers
Each Dockerfile lists the author(s)/maintainer(s) as a metadata LABEL, but the authors/maintainers of the docker images are:
- @kapsakcj
- @k-florek
- @garfinjm
- @kevinlibuit
- @erinyoung
- @lskatz
- @stjacqrm
- @AbigailShockey
- @andersgs
- @logan-fink
- @tgallagh
- @koakeson
- @joacjo
- @rpetit3
- @jvhagey
- @Antonia-Chalka
- @MillironX
- @frankambrosio3
- @HNHalstead
- @fanninpm
- @SarahNadeau
- @DOH-HNH0303
- @jwarnn
- @stitam
- @tiverson
- @sbthandras
- @idolawoye
- @michellescribner
- @cjjossart
- @jlumpe
- @kissake
- @sage-wright
- @ankushkgupta
- @kyleoconnell
- @CTindall-1
- @hkunerth
- @eetueklund
- @wchen190
- @shelby-bennett
- @cjalcorta
- @mndoucette
- @jcw349
- @poojasgupta
- @hollygene
- @HarryHung
- @sam-baird
- @eproctor118
- @cimendes
- @golden75
- @Kincekara
- @Haikelnb
- @kprus
- @arianna-smith
- @laura-bankers
- @evagunawan
- @nawrockie
- @stephenturner
- @soejun
- @taylorpaisie
- @fraser-combe