Murali-group / Beeline

BEELINE: evaluation of algorithms for gene regulatory network inference
GNU General Public License v3.0
171 stars 53 forks source link

Use Docker hub image for algorithms, instead of building from Dockerfile #47

Closed adyprat closed 3 years ago

adyprat commented 3 years ago

Building docker images from Dockerfile no longer works easily. There are several reasons for this:

  1. Possibly the biggest reason: installing older versions of software using biomanager in R is not easy (or at least unclear to me)! Many algorithms built in R are bound to fail eventually due to a lack of easy way to specify the version of the dependencies used. Moreover, many new builds can fail even if some nested dependencies of certain packages are no longer compatible (depending on how they are set-up internally). One year in, setting up Scribe from Dockerfile already doesn't work (see issue https://github.com/Murali-group/Beeline/issues/45) 🤷.
  2. Some algorithms used in BEELINE come from github repos. If for some reason, the owner of those repos updates/removes a public repo (even if they are pointed to an older commit); the docker set-up is not going to work (!).
  3. Any other system commands might pose a problem too (for example: the line RUN apt-get update in the Dockerfiles is needed to install some packages/libraries, and can potentially break things down the line).

The only way I can think of getting around this issue is to make the images built at the time of manuscript preparation available via Docker hub. While building an image using Dockerfile should work (until it wont), the initial set-up going forward is to going to pull an existing image instead of relying on Dockerfiles.

An example image is already available here: https://hub.docker.com/repository/docker/grnbeeline/scribe

Here are the steps needed to push existing images (for future reference):

  1. Login docker login --username <your username>
  2. Find image ID for the pre-built docker image using docker images
  3. Tag a local image: docker tag <image ID> <docker hub username>/<algorithm name>:<tag>
  4. Alternative to steps 2-3 is to just append your docker hub username to existing images docker tag <algorithm name>:<tag> <docker hub username>/<algorithm name>:<tag>
  5. Push to docker hub: docker push <docker hub username>/<algorithm name>:<tag>

If for some reason you need to update an existing image to create new versions:

  1. Run the docker to make changes: docker run -it <docker hub username>/<algorithm name>:<tag> /bin/bash
  2. Once inside the container, make your changes; exit the docker. Since we did not use --rm flag while running docker in the previous step, the modified container will be available for later use, even after exiting.
  3. Access the modified container id by using docker ps -a
  4. Commit modified image docker commit <container ID> <docker hub username>/<algorithm name>:<new tag>
  5. Push to docker hub: docker push <docker hub username>/<algorithm name>:<new tag>