naobservatory / mgs-pipeline

MIT License
4 stars 2 forks source link

Add new stage, samplereads #31

Closed jeffkaufman closed 8 months ago

jeffkaufman commented 8 months ago

This is to support another stage, readlengths, which isn't implemented yet.

Note that this uses streaming instead of batching, which should be faster in addition to using much less disk space (and so allow greater parallelization since we have more cores than disk)