Provide a parameter to disable rmarkdown report creation in the differential abundance pipeline

nf-core / differentialabundance

Differential abundance analysis for feature/ observation matrices from platforms such as RNA-seq

https://nf-co.re/differentialabundance

MIT License

64 stars 37 forks source link

Provide a parameter to disable rmarkdown report creation in the differential abundance pipeline #275

Open shreya-sharma20 opened 5 months ago

shreya-sharma20 commented 5 months ago

Description of feature

When analyzing a large number of samples using the differential abundance pipeline, the markdown notebook compilation step fails due to memory issues.

Error message: "Error: pandoc document conversion failed with error 9"
Process causing this error: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:RMARKDOWNNOTEBOOK

For now, I have increased the memory for this process.

Feature Request: It'll be helpful if there is a parameter to disable this process if reports are not required.

dshechter commented 5 months ago

Additional suggestion to increase the RAM allocated for this process, e.g.

process {
    withName: 'NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:RMARKDOWNNOTEBOOK' {
        memory = 200.GB
    }
}

Could work with less than 200, just haven't tried it.

rcorfixs commented 3 months ago

I have been running jobs using the attached workflow, and one of my workflows has been struggling to finish as it continues to encounter this error due to there being many samples to analyze. This study with 45 samples hasn't successfully finished the differential abundance pipeline, depite having 256 GBs of memory and 64 CPUs per task when running on an HPC. I've run jobs that have anywhere from 5 to 35 samples in the study, and those compile just fine while only having 24 GBs of memory and 8 CPUs per task. If anyone has any suggestions or potential fixes, I'd love to hear your response. RNAseq_workflow