search

phac-nml / irida

Canada’s Integrated Rapid Infectious Disease Analysis Platform for Genomic Epidemiology
https://irida.ca
Apache License 2.0
40 stars 31 forks source link

Automatically Install Tools Using ephemeris #466

Open dfornika opened 5 years ago

dfornika commented 5 years ago

Describe your idea for a new feature

It would make it easier to install new pipeline plugins if IRIDA could automatically detect and install all tools required for the pipeline to run.

Describe how you would access this feature?

Automatic, during pipeline detection and loading.

Additional information

The IRIDA documentation already describes how administrators can use ephemeris to install tools for a pipeline:

https://irida.corefacility.ca/documentation/administrator/galaxy/setup/#automated-installation-of-tools

The shed-tools install command takes a YAML-format tools-list.yml file as input that contains details about which tools to install. The same info is also available in the irida_workflow.xml file for each pipeline. Maybe IRIDA could automatically generate a tools-list.yml file and pass that to the shed-tools install command during pipeline loading?

This should probably be an opt-in feature, since not all IRIDA installations would necessarily have ephemeris installed.

dfornika commented 5 years ago

This class exists to generate a tools-list.yml file from an irida_workflow.xml:

https://github.com/phac-nml/irida/blob/development/src/main/java/ca/corefacility/bioinformatics/irida/util/ToolsListExporter.java

Maybe the functionality could be added to:

https://github.com/phac-nml/irida/blob/development/src/main/java/ca/corefacility/bioinformatics/irida/service/workflow/IridaWorkflowLoaderService.java

(?)

glwinsor commented 4 years ago

I'm interested in this too. Is there a tools-list.yml that is already made or do I need to take the irida_workflow.xml and generate it myself?

apetkau commented 4 years ago

Thanks. This is a good idea.

So, in the main IRIDA platform the tools-list.yml is generated automatically at build time by Maven by executing code in ToolsListExporter.java (see https://github.com/phac-nml/irida/blob/343790fdd3ae64570900b78e6fa1938e5a024b6a/pom.xml#L1038). This tools-list.yml is then placed in the released *.zip file we upload to GitHub.

For plugin pipelines, you could probably do something similar (e.g., configure Maven to generate a tools-list.yml file when building the JAR). However, you'd also have to make sure to load up the workflow XML file using https://github.com/phac-nml/irida/blob/development/src/main/java/ca/corefacility/bioinformatics/irida/service/workflow/IridaWorkflowLoaderService.java, so that you have the correct Java objects.

Alternatively, this could be handled by writing some separate script to convert the IRIDA XML file to a tools-list.yml file for use with ephemeris.

glwinsor commented 4 years ago

Thanks, i have a .gz release that contains a docker/virtual-machine/data/tools-list.yml. Is that the same?

apetkau commented 4 years ago

The tools-list.yml files are in the IRIDA .zip files on the releases page (https://github.com/phac-nml/irida/releases).

apetkau commented 4 years ago

It's pretty much that same as the Docker tools-list.yml, except that the one for Docker may be out of date (it was updated when we last updated the Galaxy Docker for IRIDA).

apetkau commented 4 years ago

In theory you could just write the tools-list.yml file yourself and include in each plugin. It would just have to be something people remember to keep in sync with the IRIDA workflow XML file.

glwinsor commented 4 years ago

Thanks Aaron, I found the .zip version and will try that.