Open aghr opened 7 months ago
I tried to fix this bug and could not recreate it on the dev or master branch. Therefore, I want to make sure, if I understood the problem correctly:
This issue has not had any activity since @jenmuell tried to assist in March having failed to replicate (thank you @jenmuell !).
Closing for now, but @aghr feel free to reopen if you can reproduce the error on dev
and provide us with enough information to do so.
Description of the bug
I apply the nfcore diffabundance pipeline on the output of the nfcore rnaseq pipeline on bulk RNA-seq data. The data contains >2 conditions with several replicates per condition. That's why I want to invoke the diffabundance pipeline with parameter differential_subset_to_contrast_samples: true . The pipeline crashes with several errors when invoking it with differential_subset_to_contrast_samples: true , whereas it runs with success with differential_subset_to_contrast_samples: false.
My guess on this error is that the samplesheet.csv contains all samples across >2 conditions. Setting differential_subset_to_contrast_samples: true will run a DESeq2 analysis for each contrast comparing two conditions. There are several contrasts and the output for each run will contain data only for the samples of the two conditions of the contrast. not for all samples in samplesheet.csv. It can be that later when running the GSEA on the output of each DESeq2 analysis individually GSEA also reads in the samplesheet.csv and expects all samples of the sample sheet to be contained in the output of the DESeq2 analyses but this is not the case when setting differential_subset_to_contrast_samples: true .
Command used and terminal output
Relevant files
Excerpt from .nextflow.log ########################
Feb-16 12:54:50.105 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 24; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e115); status: COMPLETED; exit: 1; error: -; workDir: /XXX/XXX/output/workflow/work/d1/4066eaca337f5ccd8c02578e84f7d8] Feb-16 12:54:50.112 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e115); work-dir=/XXX/XXX/output/workflow/work/d1/4066eaca337f5ccd8c02578e84f7d8 error [nextflow.exception.ProcessFailedException]: Process
NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e115)
terminated with an error exit status (1) Feb-16 12:54:50.139 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e115)'Caused by: Process
NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e115)
terminated with an error exit status (1)Command executed:
# Run GSEA
gsea-cli GSEA \ -res loop_vs_wt_e135.gct \ -cls loop_vs_wt_e115.cls#loop_e115_versus_wtloop_e115 \ -gmx mh.all.v2023.2.Mm.symbols.gmt \ -chip Mus_musculus.anno.feature_metadata.chip -collapse true \ -out . \ --rpt_label loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols \ -nperm 1000 -permute phenotype -scoring_scheme weighted -metric Signal2Noise -sort real -order descending -set_max 1000 -set_min 10 -norm meandiv -rnd_type no_balance -make_sets true -median false -num 100 -plot_top_x 50 -rnd_seed 42 -save_rnd_lists false -zip_report true
# Un-timestamp the outputs for path consistency mv loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols.Gsea./ . timestamp=$(cat *.rpt | grep producer_timestamp | awk '{print $2}')
for pattern in ${timestamp} .${timestamp}; do find . -name "${pattern}" | sed "s|^./||" | while read -r f; do mv $f ${f//$pattern/} done done sed -i.bak "s/[.]$timestamp//g" .rpt .html && rm *.bak
# Prefix files that currently lack it ls -p | grep -v / | grep -v "loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols." | while read -r f; do mv $f loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols.${f} sed -i.bak "s/$f/loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols.${f}/g" .rpt .html && rm *.bak done
cat <<-END_VERSIONS > versions.yml "NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA": gsea: 4.3.2 END_VERSIONS
Command exit status: 1 Command output: Using system JDK. [1708084486368] [INFO] Parameters passed to GSEA tool: [1708084486490] [INFO] gmx mh.all.v2023.2.Mm.symbols.gmt [1708084486490] [INFO] res loop_vs_wt_e135.gct [1708084486490] [INFO] cls loop_vs_wt_e115.cls#loop_e115_versus_wtloop_e115 [1708084486491] [INFO] rnd_seed 42 [1708084486491] [INFO] rpt_label loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols [1708084486491] [INFO] collapse true [1708084486492] [INFO] zip_report true [1708084486492] [INFO] gui false [1708084486492] [INFO] out . [1708084486493] [INFO] chip Mus_musculus.anno.feature_metadata.chip [1708084486493] [INFO] norm meandiv [1708084486493] [INFO] nperm 1000 [1708084486494] [INFO] permute phenotype [1708084486494] [INFO] rnd_type no_balance [1708084486494] [INFO] scoring_scheme weighted [1708084486495] [INFO] metric Signal2Noise [1708084486495] [INFO] sort real [1708084486495] [INFO] order descending [1708084486496] [INFO] make_sets true [1708084486496] [INFO] median false [1708084486496] [INFO] num 100 [1708084486497] [INFO] plot_top_x 50 [1708084486497] [INFO] rnd_seed 42 [1708084486497] [INFO] save_rnd_lists false [1708084486498] [INFO] set_max 1000 [1708084486498] [INFO] set_min 10 [1708084486549] [INFO] Made Vdb dir JIT: feb16 [1708084486577] [WARNING] Couldn't create user preferences directory. User preferences are unusable. 445,3 77%
[1708084486580] [WARNING] java.io.IOException: No such file or directory [1708084486597] [INFO] Begun importing: Dataset from: loop_vs_wt_e135.gct # of elements = 1 loop_vs_wt_e115.cls#loop_e115_versus_wtloop_e115 [1708084488043] [INFO] Begun importing: Template from: loop_vs_wt_e115.cls to parse>mh.all.v2023.2.Mm.symbols.gmt< got: [mh.all.v2023.2.Mm.symbols.gmt] [1708084488071] [INFO] Begun importing: GeneSetMatrix from: mh.all.v2023.2.Mm.symbols.gmt [1708084488089] [INFO] Using file 'mh.all.v2023.2.Mm.symbols.gmt' with unknown version [1708084488092] [INFO] Begun importing: Chip from: Mus_musculus.anno.feature_metadata.chip [1708084488177] [INFO] Parsed from dotchip : 57180 [1708084488296] [INFO] Creating collapsed dataset loop_vs_wt_e135_collapsed_to_symbols, chosen mode 0 [1708084488297] [INFO] Collapsing dataset was done. Original: 24182x6 (ann: 24182,6,Mus_musculus.anno.feature_metadata.chip) collapsed: 23988x6 (ann: 23988,6,Mus_musculus.anno.feature_metadata.chip) [1708084488379] [INFO] Extracting ds: loop_vs_wt_e135_collapsed_to_symbols by template: loop_vs_wt_e115.cls#loop_e115_versus_wtloop_e115 [1708084488408] [INFO] Renaming rpt dir on error to: ./error_loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols.Gsea.1708084488014
Command error: [1708084486491] [INFO] rpt_label loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols [1708084486491] [INFO] collapse true [1708084486492] [INFO] zip_report true [1708084486492] [INFO] gui false [1708084486492] [INFO] out . [1708084486493] [INFO] chip Mus_musculus.anno.feature_metadata.chip [1708084486493] [INFO] norm meandiv [1708084486493] [INFO] nperm 1000 [1708084486494] [INFO] permute phenotype [1708084486494] [INFO] rnd_type no_balance [1708084486494] [INFO] scoring_scheme weighted [1708084486495] [INFO] metric Signal2Noise [1708084486495] [INFO] sort real [1708084486495] [INFO] order descending [1708084486496] [INFO] make_sets true [1708084486496] [INFO] median false [1708084486496] [INFO] num 100 [1708084486497] [INFO] plot_top_x 50 [1708084486497] [INFO] rnd_seed 42 [1708084486497] [INFO] save_rnd_lists false [1708084486498] [INFO] set_max 1000 [1708084486498] [INFO] set_min 10 [1708084486549] [INFO] Made Vdb dir JIT: feb16 [1708084486577] [WARNING] Couldn't create user preferences directory. User preferences are unusable. [1708084486580] [WARNING] java.io.IOException: No such file or directory [1708084486597] [INFO] Begun importing: Dataset from: loop_vs_wt_e135.gct # of elements = 1 loop_vs_wt_e115.cls#loop_e115_versus_wtloop_e115 [1708084488043] [INFO] Begun importing: Template from: loop_vs_wt_e115.cls to parse>mh.all.v2023.2.Mm.symbols.gmt< got: [mh.all.v2023.2.Mm.symbols.gmt] [1708084488071] [INFO] Begun importing: GeneSetMatrix from: mh.all.v2023.2.Mm.symbols.gmt [1708084488089] [INFO] Using file 'mh.all.v2023.2.Mm.symbols.gmt' with unknown version [1708084488092] [INFO] Begun importing: Chip from: Mus_musculus.anno.feature_metadata.chip [1708084488177] [INFO] Parsed from dotchip : 57180 [1708084488296] [INFO] Creating collapsed dataset loop_vs_wt_e135_collapsed_to_symbols, chosen mode 0 [1708084488297] [INFO] Collapsing dataset was done. Original: 24182x6 (ann: 24182,6,Mus_musculus.anno.feature_metadata.chip) collapsed: 23988x6 (ann: 23988,6,Mus_musculus.anno.feature_metadata.chip) [1708084488379] [INFO] Extracting ds: loop_vs_wt_e135_collapsed_to_symbols by template: loop_vs_wt_e115.cls#loop_e115_versus_wtloop_e115 java.lang.ArrayIndexOutOfBoundsException: col:30 > matrix's fColCnt:6 at org.gsea_msigdb.gsea/edu.mit.broad.genome.math.Matrix.getColumnV(Matrix.java:261) at org.gsea_msigdb.gsea/edu.mit.broad.genome.objects.DefaultDataset.getColumn(DefaultDataset.java:289) at org.gsea_msigdb.gsea/edu.mit.broad.genome.objects.TemplateFactory.extract(TemplateFactory.java:85) at org.gsea_msigdb.gsea/edu.mit.broad.genome.alg.DatasetGenerators.extract(DatasetGenerators.java:292) at org.gsea_msigdb.gsea/edu.mit.broad.genome.alg.DatasetGenerators.extract(DatasetGenerators.java:288) at org.gsea_msigdb.gsea/xtools.gsea.AbstractGsea2Tool.execute_one(AbstractGsea2Tool.java:79) at org.gsea_msigdb.gsea/xtools.gsea.AbstractGsea2Tool.execute_one_with_reporting(AbstractGsea2Tool.java:103) at org.gsea_msigdb.gsea/xtools.gsea.Gsea.execute(Gsea.java:166) at org.gsea_msigdb.gsea/xtools.api.AbstractTool.module_main(AbstractTool.java:401) at org.gsea_msigdb.gsea/org.genepattern.modules.GseaWrapper.main(GseaWrapper.java:291) at org.gsea_msigdb.gsea/xapps.gsea.CLI.main(CLI.java:29) [1708084488408] [INFO] Renaming rpt dir on error to: ./error_loop_vs_wt_e115.mh.all.v2023.2.Mm.symbols.Gsea.1708084488014
Work dir: /XXX/XXX/output/workflow/work/d1/4066eaca337f5ccd8c02578e84f7d8
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named
.command.sh
Feb-16 12:54:50.147 [Task monitor] INFO nextflow.Session - Execution cancelled -- Finishing pending tasks before exit Feb-16 12:54:50.183 [main] DEBUG nextflow.Session - Session await > all processes finished Feb-16 12:54:50.185 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 22; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (inv_vs_wt_e125); status: COMPLETED; exit: 1; error: -; workDir: /XXX/XXX/output/workflow/work/f8/e014b5add0e042e4c5a6962151379e] Feb-16 12:54:50.185 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (inv_vs_wt_e125); work-dir=/XXX/XXX/output/workflow/work/f8/e014b5add0e042e4c5a6962151379e error [nextflow.exception.ProcessFailedException]: ProcessNFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (inv_vs_wt_e125)
terminated with an error exit status (1) Feb-16 12:54:50.186 [Actor Thread 21] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:RMARKDOWNNOTEBOOK; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Feb-16 12:54:50.190 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 21; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e125); status: COMPLETED; exit: 1; error: -; workDir: /XXX/XXX/output/workflow/work/0d/ed7a8ba1a40875e9e363aca7f0b884] Feb-16 12:54:50.191 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e125); work-dir=/XXX/XXX/output/workflow/work/0d/ed7a8ba1a40875e9e363aca7f0b884 error [nextflow.exception.ProcessFailedException]: ProcessNFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e125)
terminated with an error exit status (1) Feb-16 12:54:50.191 [Actor Thread 29] DEBUG nextflow.file.FileCollector - Deleting file collector temp dir: /tmp/nxf-173997325202795786 Feb-16 12:54:50.195 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 23; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (inv_vs_wt_e135); status: COMPLETED; exit: 1; error: -; workDir: /XXX/XXX/output/workflow/work/0f/b8b5e11d87680d95e935b465cc1550] Feb-16 12:54:50.195 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (inv_vs_wt_e135); work-dir=/XXX/XXX/output/workflow/work/0f/b8b5e11d87680d95e935b465cc1550 error [nextflow.exception.ProcessFailedException]: ProcessNFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (inv_vs_wt_e135)
terminated with an error exit status (1) Feb-16 12:54:50.211 [Actor Thread 36] DEBUG nextflow.sort.BigSort - Sort completed -- entries: 7; slices: 1; internal sort time: 0.022 s; external sort time: 0.002 s; total time: 0.024 s Feb-16 12:54:50.222 [Actor Thread 36] DEBUG nextflow.file.FileCollector - Saved collect-files list to: /XXX/XXX/output/workflow/work/collect-file/a0e32014125d264c06fdb3864c98f054 Feb-16 12:54:50.231 [Actor Thread 36] DEBUG nextflow.file.FileCollector - Deleting file collector temp dir: /tmp/nxf-8002081208300576428 Feb-16 12:54:50.248 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 25; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e135); status: COMPLETED; exit: 1; error: -; workDir: /XXX/XXX/output/workflow/work/25/fdb2744533edd1885e5d295430fd17] Feb-16 12:54:50.248 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e135); work-dir=/XXX/XXX/output/workflow/work/25/fdb2744533edd1885e5d295430fd17 @ error [nextflow.exception.ProcessFailedException]: ProcessNFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GSEA_GSEA (loop_vs_wt_e135)
terminated with an error exit status (1) Feb-16 12:54:55.869 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 20; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:PLOT_EXPLORATORY (condition); status: COMPLETED; exit: 1; error: -; workDir: /XXX/XXX/output/workflow/work/35/15b254f67de962bd2a30119d28fa04] Feb-16 12:54:55.870 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:PLOT_EXPLORATORY (condition); work-dir=/XXX/XXX/output/workflow/work/35/15b254f67de962bd2a30119d28fa04 error [nextflow.exception.ProcessFailedException]: ProcessNFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:PLOT_EXPLORATORY (condition)
terminated with an error exit status (1) Feb-16 12:54:59.187 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 16; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:DESEQ2_DIFFERENTIAL ([id:inv_vs_wt_e125, variable:condition, reference:wtinv_e125, target:inv_e125, blocking:]); status: COMPLETED; exit: 0; error: -; workDir: /XXX/XXX/output/workflow/work/a4/da4d864c8b74d80640091386c57c4e] Feb-16 12:54:59.219 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:DESEQ2_DIFFERENTIAL > Skipping output binding because one or more optional files are missing: fileoutparam<5:1> Feb-16 12:55:01.812 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 19; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:PLOT_DIFFERENTIAL (loop_vs_wt_e125); status: COMPLETED; exit: 0; error: -; workDir: /XXX/XXX/output/workflow/work/06/3b88df1f289283881b4c884e53d0ff] Feb-16 12:55:01.827 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:PLOT_DIFFERENTIAL > Skipping output binding because one or more optional files are missing: fileoutparam<1:1> Feb-16 12:55:02.403 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 18; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:PLOT_DIFFERENTIAL (loop_vs_wt_e115); status: COMPLETED; exit: 0; error: -; workDir: /XXX/XXX/output/workflow/work/eb/299f44d5163890210f3e5e37d04dcb]Feb-16 12:55:11.713 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:DESEQ2_DIFFERENTIAL > Skipping output binding because one or more optional files are missing: fileoutparam<5:1> Feb-16 12:55:11.720 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop Feb-16 12:55:11.720 [main] DEBUG nextflow.Session - Session await > all barriers passed Feb-16 12:55:11.783 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'PublishDir' shutdown completed (hard=false) Feb-16 12:55:11.828 [main] INFO nextflow.Nextflow - -^[[0;35m[nf-core/differentialabundance]^[[0;31m Pipeline completed with errors^[[0m- Feb-16 12:55:11.835 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=20; failedCount=6; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=37m 26s; failedDuration=35.8s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=11; peakCpus=36; peakMemory=82 GB; ] Feb-16 12:55:11.835 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file Feb-16 12:55:11.838 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report Feb-16 12:55:12.541 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline Feb-16 12:55:12.807 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done Feb-16 12:55:12.807 [main] INFO org.pf4j.AbstractPluginManager - Stop plugin 'nf-validation@1.1.3' Feb-16 12:55:12.807 [main] DEBUG nextflow.plugin.BasePlugin - Plugin stopped nf-validation Feb-16 12:55:12.822 [main] DEBUG nextflow.util.ThreadPoolManager - Thread pool 'FileTransfer' shutdown completed (hard=false) Feb-16 12:55:12.822 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye ~
System information
nextflow version 23.10.1.5891 nf-core/differentialabundance v1.4.0-ga3d664c Hardware: Desktop LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.7.1908 (Core) Release: 7.7.1908 Codename: Core Executor: local Container: Singularity