populationgenomics / analysis-runner

MIT License
2 stars 4 forks source link

Reduce cromwell polling #658

Closed MattWellie closed 11 months ago

MattWellie commented 11 months ago

Allows a pass-thru argument for the max polling interval in the cromwell watcher jobs.

I've found high numbers of simultaneous jobs are demolishing the Cromwell API, and a longer pause between re-polling may alleviate this. I'd expect to overload this value on the production-pipelines side to reflect the jobs being run. A poll time of 10 mins could be more appropriate for a job expected to run for ~18 hours, instead of 1 per minute