search

CoBrALab / qbatch

The Unlicense
27 stars 13 forks source link

Implement --min-chunksize #154

Open gdevenyi opened 6 years ago

gdevenyi commented 6 years ago

If I have something like a joblist of 81 commands and my chunksize is 20, I get this unfortunate result of jobs with 40,40,1. This is bad for the cluster.

Should be able to define a tolerance of number of commands (percent?) for overpacking the last job. Requires a little bit of extra logic to handle it but probably worth the effort.

pipitone commented 6 years ago

How about a --min-chunksize option defaulting to 1. If the last chunk size is < the min-chunk size, you add those tasks to the previous chunk.

gdevenyi commented 6 years ago

Ah, that's a much clearer naming than tolerance :+1: