Improve performance of Mageck MLE module

MPUSP / nf-core-crispriscreen

Process next generation sequencing data obtained from CRISPRi repression library screenings

MIT License

4 stars 2 forks source link

Improve performance of Mageck MLE module #23

Closed m-jahn closed 7 months ago

m-jahn commented 1 year ago

Description of feature

Mageck currently is comparatively slow compared with DESeq2-based fitness calculation
although it uses all allocated resources, processing time is 20x longer than for DESeq2
example dataset with 12 samples, 4 comparisons, 100,000 reads per sample takes >10 min with 12 CPU cores
DESeq2 takes 2-3 min with 1 CPU core

m-jahn commented 1 year ago

No development on this issue from my side but problem is persisting somehow. I noticed however that the NF Mageck module at least makes good use of the computational resources, i.e. uses all assigned cores. Maybe Mageck itself is just slow and nothing can be done without altering the underlying code.

I'm tagging @LaurenceKuhl here who mentioned that there would be nextflow tricks to improve performance (https://github.com/MPUSP/nf-core-crispriscreen/issues/22#issuecomment-1473292889).

m-jahn commented 7 months ago

does not seem like there can be easy improvements of Mageck performance. closing this for now