mllg / batchtools

Tools for computation on batch systems
https://mllg.github.io/batchtools/
GNU Lesser General Public License v3.0
169 stars 51 forks source link

[Question] Dynamically change number of concurrent jobs #270

Open dipterix opened 3 years ago

dipterix commented 3 years ago

Hi, Thanks for making this great package. I'm writing code to run multiple registries at the same time. Some tasks (collections of jobs) finish prior to the others. I was wondering if is there any way to release/balance/cancel resources dynamically

For example

If Task 1 finish earlier than the Task 2, is there a way I can increase the max concurrent jobs to 20 for task 2 and cancel task 3 for those which is still submitting?

One way I come up with is we could monitor certain files within the registry folder, if those files exist, then dynamically change the registry settings for jobs that haven't been submitted yet and then remove those files