marbl / verkko

265 stars 27 forks source link

RED memory usage is high on simplex reads #220

Closed brianwalenz closed 6 months ago

brianwalenz commented 6 months ago

Verkko makes RED batches with exactly 20,000 reads in them. for long simplex reads, this exhausts (default) memory limits.

Adding a parameter to set 'red_total' or something like that instead of the hardcoded len=19999 at c4-findErrors.sm line 46 would work. But it's not that much work to do actual partitioning based on read length. Ideally we'd set the max based on red_memory but that's been a headache in canu to get the scaling correct...