search

ewels / clusterflow

A pipelining tool to automate and standardise bioinformatics analyses on cluster environments.
https://ewels.github.io/clusterflow/
GNU General Public License v3.0
97 stars 27 forks source link

Sum queued download tasks #64

Closed ewels closed 9 years ago

ewels commented 9 years ago

Instead of --qstat showing hundreds of download jobs all waiting for dependencies, just count these and show a summary.

ewels commented 9 years ago

These changes broke cf --qstat - if there are multiple pipelines running, the headers are shown all together in a group at the top. Needs fixing.

ewels commented 9 years ago

Worse than just this - I think that some jobs now aren't being shown, if they're running at the same hierarchy level? eg. fastqc and trim_galore

ewels commented 9 years ago

..and non Cluster Flow jobs now aren't shown. This is all kinds of broken.

ewels commented 9 years ago

Reverted above changes in fe5727e08789ead6417b29bb3b3ba3e1d740ec55

Will come back and try again later..

ewels commented 9 years ago

This is surprisingly difficult to do given the way that these parsing functions recursively iterate through the jobs hash. As this isn't a very common problem, I'm going to abandon this for now.

For a bonus, I did to some tidying and add a new feature (summary counts): 84d8a77fbbf5699f1fd0cd27c5ab356e894fb97b