galaxyproject / ephemeris

Library for managing Galaxy plugins - tools, index data, and workflows.
https://ephemeris.readthedocs.org/
Other
27 stars 38 forks source link

Run-data-managers only runs one job at a time. #54

Closed rhpvorderman closed 6 years ago

rhpvorderman commented 6 years ago

Not very useful if installing 18 genomes at once with 5 data managers. Waiting on 90 jobs one at a time. Since data managers may be interdependent it is maybe best to run these jobs per data manager. So starting 18 jobs in parallel for each data manager and then wait until all jobs are finished to reload the data table and start the next data manager for the 18 genomes.