search

mllg / batchtools

Tools for computation on batch systems
https://mllg.github.io/batchtools/
GNU Lesser General Public License v3.0
170 stars 51 forks source link

Implement sharding #217

Open mllg opened 5 years ago

mllg commented 5 years ago

This reduces the burden on the filesystem for large studies.

mllg commented 5 years ago

Maybe even look into making the file system operations abstract, so we can switch them out with a network proto.