cokelaer / damona

singularity environment manager (application to NGS and bioinformatics)
https://damona.readthedocs.io
BSD 3-Clause "New" or "Revised" License
5 stars 2 forks source link

design change #36

Closed cokelaer closed 2 months ago

cokelaer commented 2 months ago

right now, we have a registry where we add binaries. For instance, we can have a registry for kraken that also install lots of other software. So, there is redundancy between several software that may install the same software in different container. That a feature. This means that blast can be added as binary in several registry.

how to cope with that on the long term, especially, the versioning. if you have blast1.0 in container 1 and blast2.0 in container 2, they will clash. users do not know which one overwrites the other and so on.

one idea is to have dedicated software/registry per tool. of course, this is lots of works. right now we have 80 containers but 500 tools so this means we should create an additional 420 containers which is quite redundant.

Instead, we could imagine the following design:

if blast is in a container, there is no need to recreat a singularity image/container. Just reuse the existing container that already providse the tools. For instance blast is in the kraken2 cotnainer so we could just have a registry for blast as follows:

blast:
   doi: 10.5281/zenodo.5801359
  zenodo_id: 5801360
 releases:
    1.0.0:
       download: https://zenodo.org/record/5801365/files/kraken_2.0.9.img
       md5sum: bc195a921148969c7f9482f9fe25b3cd
      doi: 10.5281/zenodo.5801365
      filesize: 723301918
      binaries: blast

we need to think about doi and so on though. problems:

  1. here there is no need to upload a container since it exists already but how to handle the DOI of the blast registry then ?
  2. besides, even though the container is already installed with kraken, if we install blast, it will redownload the file again, naming it blast.img along side the same exisitng file kraken.img. e should use aliases/sym link
cokelaer commented 2 months ago

Quite easy fix finally. Just use the container name instead of registry name; Available in v