mvdbeek
commented
4 years ago A high-level view for building containers for Galaxy tools is this: If it is a single package requirement in bioconda, the bioconda build process will publish a container. If it is a single package requirement from conda-forge, or if multiple-package are required we use planemo-monitor against a list of known galaxy tool repos (any repo can be added to https://github.com/galaxyproject/planemo-monitor) to create PRs against https://github.com/BioContainers/multi-package-containers/, which in turn will build and publish those containers. This has some limitations, see below.
Some of the IUC tools listed here are not the latest version (at least bedtools), I don't think we have the resources to build and make sure containers work for old versions, at least at this point. Once things are smooth for all the latest versions we could consider running planemo-monitor against the tools on CVMFS.
A quick look at the converters suggests they don't have an annotated requirement, so that'd be a first step, that's a Galaxy issue. Once they do they will be built through planemo-monitor (https://github.com/galaxyproject/planemo-monitor/blob/master/repositories01.list#L1).
We also need to roll out a new planemo version that will build single-requirement containers and other improvements, but that's depending on https://github.com/galaxyproject/planemo/pull/978, which probably would build the rest of the IUC tools on your list that are at the latest version but that are not in https://github.com/galaxyproject/tools-iuc/issues/2701.
For non-IUC tools you could run planemo-monitor yourself (if you install planemo from master), but I think you should wait until we have a new planemo release, since the build process would probably result in many slightly broken containers right now (https://github.com/BioContainers/multi-package-containers/issues/940 for some things that are still left to do).
I was excited to try out the new dependency resolution UI in 20.01 but it also shined a light on just how many tools do not have a container to resolve to. Out of 1,660 tools available on the CVMFS, 570 do not resolve (34%). Admittedly, many are from non-IUC repos but I'm wondering what is the missing link and how can we increase the number of resolved containers?
Below is a list of currently unresolved containers. It would be helpful if those familiar with the tools could please edit the list to cross out the tools that are known not working or not necessary and ideally highlight the popular ones.
Further, could someone please detail the steps necessary to fix one of these tools? Sorry for a high-level question but I'm (still) not sure how the generic automated tool containerization pipeline works, especially when it comes to non-IUC repos.
Finally, I manually cross-searched a few tools listed in https://github.com/galaxyproject/tools-iuc/issues/2701, but didn't run into any overlap. I'm guessing the tools listed below will show up as missing tests only after a corresponding container has been created?