wdecoster / chopper

MIT License
135 stars 11 forks source link

Configure global thread pool; use atomics instead of mutex #11

Closed jvkersch closed 1 year ago

jvkersch commented 1 year ago

My apologies for not opening an issue first; this PR arose out of some experimentation and I thought it would be easier to describe the issue with the code to refer to.

This PR does two things:

This is somewhat of an experimental approach, but I see the following timings (macOS, m1) while filtering a 400 MB fastq file with differing numbers of threads (time target/debug/chopper -l 200 -q 7 --threads $N < in.fastq > out.fastq). Specifically, I looked at 1, 2, 4, 8 threads and recorded the following timings:

wdecoster commented 1 year ago

Awesome, thank you, I will follow up soon, after my holidays.

wdecoster commented 1 year ago

Looks good to me :) thanks for the improvements!