Closed e-tomato closed 1 year ago
SvitlanaLukicheva
commented
1 year ago Dear @e-tomato ,
I don't think this is an error, it looks like monitoring messages. It is normal for the GRNBoost process to take a long time to complete.
Did you execute a test run on our example data? Moreover, I see that you are using an outdated version of MINI-EX, I suggest to pull the changes from the repository to benefit from the latest features.
e-tomato
commented
1 year ago Hi @SvitlanaLukicheva,
Thanks for your remind. I have now updated to your MINI-EX version 2.
Then I run a test with the example data. This time MINI-EX looks happier than before, however it quickly said "goodbye" to me after only excuting the 'get_expressed_genes', 'unzip_motifMappings', and 'get_topDEGs ' functions. No any new error messages were shown except the above mentioned monitoring messages. It looks like that the GRNBoost process was not initiated.
Guojian
Command: module load bioinfo/Nextflow-v21.10.6 module load system/singularity-3.7.3
nextflow -C miniex_run_example.config run miniex.nf
miniex_run_example.config:
executor {
name = 'slurm'
queueSize = 5
}
process.container = "vibpsb/mini-ex:latest"
singularity {
enabled = true
cacheDir = "singularity_cache"
autoMounts = true
}
params {
expressionMatrix = "$baseDir/example/INPUTS/*_matrix.txt"
markersOut = "$baseDir/example/INPUTS/*_allMarkers.txt"
cell2clusters = "$baseDir/example/INPUTS/*_cells2clusters.txt"
cluster2ident = "$baseDir/example/INPUTS/*_identities.txt"
TF_list = "$baseDir/example/INPUTS/TF_list.txt"
termsOfInterest = "$baseDir/example/INPUTS/GOsIwant.txt"
//termsOfInterest = null
grnboostOut = "/$baseDir/example/OUTPUTS/GRNBoost2_output/*_grnboost2.txt"
//grnboostOut = null
doMotifAnalysis = true // set to <false> if no motif mapping data is available [CAUTION: without motif data MINI-EX is less reliable]
featureFile_motifs = "$baseDir/data_ath/ath_2021.1_motifMapping.out.gz"
infoTF = "$baseDir/data_ath/ath_TF2fam2mot.txt"
GOfile = "$baseDir/data_ath/ath_full_BP_expcur_ext_names.txt" //set to <null> if GO data is not available (when doing so, termsOfInterest should also be set to <null>)
alias = "$baseDir/data_ath/ath_gene_aliases.txt"
script_enricher = "$baseDir/bin/enricherv2.4"
script_checkInput = "$baseDir/bin/MINIEX_checkInput.py"
script_grnboost = "$baseDir/bin/MINIEX_grnboostMultiprocess.py"
script_motifs = "$baseDir/bin/MINIEX_filterForMotifs.py"
script_topDEGs = "$baseDir/bin/MINIEX_selectTopDEGs.py"
script_expTFs = "$baseDir/bin/MINIEX_filterForTFExp.py"
script_info = "$baseDir/bin/MINIEX_makeInfoFile.py"
script_clustermap = "$baseDir/bin/MINIEX_clustermap.py"
script_networkCentrality = "$baseDir/bin/MINIEX_network_analysis.py"
script_checkReference = "$baseDir/bin/MINIEX_checkRef.py"
script_filesEnrichment = "$baseDir/bin/MINIEX_makeFilesEnrichment.py"
script_makedfRef = "$baseDir/bin/MINIEX_makeRankingDf_ref.py"
script_makedfStd = "$baseDir/bin/MINIEX_makeRankingDf_std.py"
script_makeborda = "$baseDir/bin/MINIEX_makeBorda.py"
script_scoreEdges = "$baseDir/bin/MINIEX_scoreEdges.py"
script_heatmapTops = "$baseDir/bin/MINIEX_visual_heatmap_top150.py"
script_regmaps = "$baseDir/bin/MINIEX_regmap.py"
tops = "700"
expressionFilter = "10"
motifFilter = "TF-F_motifs" // To use the motifs of the TF family, specify "TF-F_motifs". To only use the motifs known for a TF, specify "TF_motifs".
topRegs = "150"
}
process {
withName: check_input_files {
clusterOptions = '--mem=50G -c 5'
}
withName: get_expressed_genes {
clusterOptions = '--mem=2G'
}
withName: run_grnboost {
clusterOptions = '--mem=15G -c 5'
}
withName: unzip_motifMappings {
executor='local'
}
withName: run_enricher_motifs {
clusterOptions = '--mem=4G'
}
withName: filter_motifs {
clusterOptions = '--mem=20G'
}
withName: filter_motifs_dummy {
executor='local'
}
withName: get_topDEGs {
clusterOptions = '--mem=10G'
}
withName: run_enricher_cluster {
clusterOptions = '--mem=4G'
}
withName: filter_expression {
clusterOptions = '--mem=20G'
}
withName: make_info_file {
clusterOptions = '--mem=10G'
}
withName: network_centrality {
clusterOptions = '--mem=20G'
}
withName: getFiles_enrichment {
clusterOptions = '--mem=10G'
}
withName: GO_enricher {
clusterOptions = '--mem=4G'
}
withName: check_reference {
clusterOptions = '--mem=10G'
}
withName: ranking_df_ref {
clusterOptions = '--mem=10G'
}
withName: ranking_df_std {
clusterOptions = '--mem=10G'
}
withName: makeBorda {
clusterOptions = '--mem=10G'
}
withName: scoreEdges {
clusterOptions = '--mem=15G'
}
withName: heatmap_tops {
clusterOptions = '--mem=20G'
}
withName: clustermap_regs {
clusterOptions = '--mem=20G'
}
withName: regmaps {
clusterOptions = '--mem=20G'
}
}
out message
N E X T F L O W ~ version 21.10.6 Launching
miniex.nf[jolly_ardinghelli] - revision: 7d53c557b4 Motif-Informed Network Inference from gene EXpression v.2.0Running TF motif enrichment filtering on TF-F_motifs Running single-cell cluster enrichment using the top 700 upregulated genes per cluster Filtering out regulons of single-cell clusters where the TF is expressed in less than 10 % of the cells Plotting expression specificity and DE calls for the top 150 regulons ...... executor > slurm (2), local (1) [- ] process > check_input_files - [74/e72bab] process > get_expressed_genes (1) [100%] 1 of 1 ✔ [e3/1f256d] process > unzip_motifMappings [100%] 1 of 1 ✔ [- ] process > run_enricher_motifs - [- ] process > filter_motifs - [01/79535e] process > get_topDEGs (1) [100%] 1 of 1 ✔ [- ] process > run_enricher_cluster - [- ] process > filter_expression - [- ] process > make_info_file - [- ] process > clustermap_regs - [- ] process > network_centrality - [- ] process > getFiles_enrichment - [- ] process > GO_enricher - [- ] process > check_reference - [- ] process > ranking_df_ref - [- ] process > makeBorda - [- ] process > scoreEdges - [- ] process > heatmap_tops - [- ] process > regmaps - [miniexExample, /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/miniexExample_matrix.txt] Done!
nextflow.log
Jul-20 12:27:21.572 [main] DEBUG nextflow.cli.Launcher - $> nextflow -C miniex_run_example.config run miniex.nf Jul-20 12:27:21.745 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 21.10.6 Jul-20 12:27:21.798 [main] INFO nextflow.cli.CmdRun - Launching
miniex.nf[jolly_ardinghelli] - revision: 7d53c557b4 Jul-20 12:27:21.824 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/miniex_run_example.config Jul-20 12:27:21.860 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile:standardJul-20 12:27:23.490 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/hguojian/.nextflow/plugins Jul-20 12:27:23.492 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[] Jul-20 12:27:23.511 [main] INFO org.pf4j.DefaultPluginStatusProvider - Enabled plugins: [] Jul-20 12:27:23.512 [main] INFO org.pf4j.DefaultPluginStatusProvider - Disabled plugins: [] Jul-20 12:27:23.518 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode Jul-20 12:27:23.535 [main] INFO org.pf4j.AbstractPluginManager - No plugins Jul-20 12:27:23.628 [main] DEBUG nextflow.Session - Session uuid: 21a06a96-b167-4dfb-a686-68753b011cc1 Jul-20 12:27:23.628 [main] DEBUG nextflow.Session - Run name: jolly_ardinghelli Jul-20 12:27:23.629 [main] DEBUG nextflow.Session - Executor pool size: 4 Jul-20 12:27:23.648 [main] DEBUG nextflow.cli.CmdRun - Version: 21.10.6 build 5660 Created: 21-12-2021 16:55 UTC (17:55 CEST) System: Linux 3.10.0-1160.el7.x86_64 Runtime: Groovy 3.0.9 on OpenJDK 64-Bit Server VM 1.8.0_262-b10 Encoding: UTF-8 (UTF-8) Process: 55137@node116 [192.168.1.116] CPUs: 4 - Mem: 251.6 GB (198.5 GB) - Swap: 0 (0) Jul-20 12:27:23.685 [main] DEBUG nextflow.Session - Work-dir: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/work [gpfs] Jul-20 12:27:23.720 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[] Jul-20 12:27:23.739 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory Jul-20 12:27:23.879 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 5; maxThreads: 1000 Jul-20 12:27:24.018 [main] DEBUG nextflow.Session - Session start invoked Jul-20 12:27:26.981 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution Jul-20 12:27:27.009 [main] INFO nextflow.Nextflow - Motif-Informed Network Inference from gene EXpression v.2.0Running TF motif enrichment filtering on TF-F_motifs Running single-cell cluster enrichment using the top 700 upregulated genes per cluster Filtering out regulons of single-cell clusters where the TF is expressed in less than 10 % of the cells Plotting expression specificity and DE calls for the top 150 regulons
Jul-20 12:27:27.193 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 5; maxThreads: 1000 Jul-20 12:27:27.323 [main] DEBUG nextflow.script.ProcessConfig - Config settings
withName:check_input_filesmatches process check_input_files Jul-20 12:27:27.331 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.331 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.347 [main] DEBUG nextflow.executor.Executor - [warm up] executor > slurm Jul-20 12:27:27.357 [main] DEBUG n.processor.TaskPollingMonitor - Creating task monitor for executor 'slurm' > capacity: 5; pollInterval: 5s; dumpInterval: 5m Jul-20 12:27:27.363 [main] DEBUG n.executor.AbstractGridExecutor - Creating executor 'slurm' > queue-stat-interval: 1m Jul-20 12:27:27.551 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:get_expressed_genesmatches process get_expressed_genes Jul-20 12:27:27.555 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.555 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.575 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:unzip_motifMappingsmatches process unzip_motifMappings Jul-20 12:27:27.577 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local Jul-20 12:27:27.577 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Jul-20 12:27:27.580 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local Jul-20 12:27:27.584 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=4; memory=251.6 GB; capacity=5; pollInterval=100ms; dumpInterval=5m Jul-20 12:27:27.596 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:run_enricher_motifsmatches process run_enricher_motifs Jul-20 12:27:27.598 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.598 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.606 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:filter_motifsmatches process filter_motifs Jul-20 12:27:27.608 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.608 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.619 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:get_topDEGsmatches process get_topDEGs Jul-20 12:27:27.620 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.620 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.852 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:run_enricher_clustermatches process run_enricher_cluster Jul-20 12:27:27.854 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.854 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.890 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:filter_expressionmatches process filter_expression Jul-20 12:27:27.892 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.892 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.929 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:make_info_filematches process make_info_file Jul-20 12:27:27.930 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.930 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.949 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:clustermap_regsmatches process clustermap_regs Jul-20 12:27:27.950 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.950 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.956 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:network_centralitymatches process network_centrality Jul-20 12:27:27.957 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.957 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.983 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:getFiles_enrichmentmatches process getFiles_enrichment Jul-20 12:27:27.984 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.984 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:27.997 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:GO_enrichermatches process GO_enricher Jul-20 12:27:27.998 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:27.998 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:28.008 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:check_referencematches process check_reference Jul-20 12:27:28.010 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:28.010 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:28.031 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:ranking_df_refmatches process ranking_df_ref Jul-20 12:27:28.032 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:28.032 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:28.043 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:makeBordamatches process makeBorda Jul-20 12:27:28.044 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:28.044 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:28.057 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:scoreEdgesmatches process scoreEdges Jul-20 12:27:28.058 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:28.058 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:28.066 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:heatmap_topsmatches process heatmap_tops Jul-20 12:27:28.067 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:28.067 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:28.076 [main] DEBUG nextflow.script.ProcessConfig - Config settingswithName:regmapsmatches process regmaps Jul-20 12:27:28.076 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: slurm Jul-20 12:27:28.076 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'slurm' Jul-20 12:27:28.080 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: clustermap_regs, filter_expression, check_input_files, ranking_df_ref, makeBorda, make_info_file, get_topDEGs, run_enricher_motifs, network_centrality, getFiles_enrichment, unzip_motifMappings, regmaps, run_grnboost, run_enricher_cluster, GO_enricher, scoreEdges, filter_motifs, filter_motifs_dummy, heatmap_tops, get_expressed_genes, ranking_df_std, check_reference Jul-20 12:27:28.083 [main] DEBUG nextflow.Session - Ignite dataflow network (38) Jul-20 12:27:28.103 [PathVisitor-1] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/; pattern: _matrix.txt; options: [:] Jul-20 12:27:28.103 [PathVisitor-3] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/; pattern: _cells2clusters.txt; options: [:] Jul-20 12:27:28.103 [PathVisitor-4] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/; pattern: _identities.txt; options: [:] Jul-20 12:27:28.103 [PathVisitor-2] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/; pattern: _allMarkers.txt; options: [:] Jul-20 12:27:28.110 [PathVisitor-7] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/OUTPUTS/GRNBoost2_output/; pattern: _grnboost2.txt; options: [:] Jul-20 12:27:28.117 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > check_input_files Jul-20 12:27:28.119 [PathVisitor-6] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/; pattern: _matrix.txt; options: [:] Jul-20 12:27:28.120 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > get_expressed_genes Jul-20 12:27:28.120 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > unzip_motifMappings Jul-20 12:27:28.120 [PathVisitor-10] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/OUTPUTS/GRNBoost2_output/; pattern: _grnboost2.txt; options: [:] Jul-20 12:27:28.125 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > run_enricher_motifs Jul-20 12:27:28.126 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > filter_motifs Jul-20 12:27:28.126 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > get_topDEGs Jul-20 12:27:28.125 [PathVisitor-10] DEBUG nextflow.file.PathVisitor - No such file: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/OUTPUTS/GRNBoost2_output/ -- Skipping visit Jul-20 12:27:28.129 [PathVisitor-8] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/; pattern: _allMarkers.txt; options: [:] Jul-20 12:27:28.134 [PathVisitor-7] DEBUG nextflow.file.PathVisitor - No such file: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/OUTPUTS/GRNBoost2_output/ -- Skipping visit Jul-20 12:27:28.133 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > run_enricher_cluster Jul-20 12:27:28.161 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > filter_expression Jul-20 12:27:28.161 [PathVisitor-1] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/; pattern: _cells2clusters.txt; options: [:] Jul-20 12:27:28.163 [PathVisitor-8] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/INPUTS/; pattern: _identities.txt; options: [:] Jul-20 12:27:28.163 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > make_info_file Jul-20 12:27:28.165 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > clustermap_regs Jul-20 12:27:28.165 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > network_centrality Jul-20 12:27:28.166 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > getFiles_enrichment Jul-20 12:27:28.168 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > GO_enricher Jul-20 12:27:28.168 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > check_reference Jul-20 12:27:28.169 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > ranking_df_ref Jul-20 12:27:28.170 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > makeBorda Jul-20 12:27:28.171 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > scoreEdges Jul-20 12:27:28.171 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > heatmap_tops Jul-20 12:27:28.171 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > regmaps Jul-20 12:27:28.171 [main] DEBUG nextflow.script.ScriptRunner - > Await termination Jul-20 12:27:28.171 [main] DEBUG nextflow.Session - Session await Jul-20 12:27:28.365 [Actor Thread 32] DEBUG nextflow.container.SingularityCache - Singularity found local store for image=docker://vibpsb/mini-ex:latest; path=/work/hguojian/snRNA-seq/MINI-EX/MINI-EX/singularity_cache/vibpsb-mini-ex-latest.img Jul-20 12:27:28.559 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Jul-20 12:27:28.569 [Task submitter] INFO nextflow.Session - [e3/1f256d] Submitted process > unzip_motifMappings Jul-20 12:27:28.582 [Task submitter] DEBUG nextflow.executor.GridTaskHandler - [SLURM] submitted process get_topDEGs (1) > jobId: 49633183; workDir: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/work/01/79535e4979c95fad7ba81f17808a1a Jul-20 12:27:28.590 [Task submitter] INFO nextflow.Session - [01/79535e] Submitted process > get_topDEGs (1) Jul-20 12:27:28.637 [Task submitter] DEBUG nextflow.executor.GridTaskHandler - [SLURM] submitted process get_expressed_genes (1) > jobId: 49633184; workDir: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/work/74/e72baba8af4348eb471afa612f0c6b Jul-20 12:27:28.637 [Task submitter] INFO nextflow.Session - [74/e72bab] Submitted process > get_expressed_genes (1) Jul-20 12:27:30.301 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: unzip_motifMappings; status: COMPLETED; exit: 0; error: -; workDir: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/work/e3/1f256d9697f8a28e1469f665fdbd2f] Jul-20 12:27:52.399 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[jobId: 49633184; id: 2; name: get_expressed_genes (1); status: COMPLETED; exit: 0; error: -; workDir: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/work/74/e72baba8af4348eb471afa612f0c6b started: 1689848872389; exited: 2023-07-20T10:27:50.778238Z; ] Jul-20 12:27:57.365 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[jobId: 49633183; id: 1; name: get_topDEGs (1); status: COMPLETED; exit: 0; error: -; workDir: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/work/01/79535e4979c95fad7ba81f17808a1a started: 1689848872370; exited: 2023-07-20T10:27:54.072199Z; ] Jul-20 12:27:57.383 [main] DEBUG nextflow.Session - Session await > all process finished Jul-20 12:28:02.364 [main] DEBUG nextflow.Session - Session await > all barriers passed Jul-20 12:28:02.370 [main] INFO nextflow.Nextflow - Done! Jul-20 12:28:02.380 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=3; failedCount=0; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=6.6s; failedDuration=0ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=2; peakCpus=2; peakMemory=0; ] Jul-20 12:28:02.453 [main] DEBUG nextflow.CacheDB - Closing CacheDB done Jul-20 12:28:02.469 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
SvitlanaLukicheva
commented
1 year ago Dear @e-tomato ,
It is normal that the GRNBoost process was not initiated as in your config file you are using a pre-computed GRNBoost output:
grnboostOut = "/$baseDir/example/OUTPUTS/GRNBoost2_output/*_grnboost2.txt".
In order to initiate it, you should set this parameter to null.
Now, it is not clear why the pipeline stops without executing all the processes without looking at your files. In the log file, I see the message No such file: /work/hguojian/snRNA-seq/MINI-EX/MINI-EX/example/OUTPUTS/GRNBoost2_output/ -- Skipping visit, probably one of the files is missing? This could explain why the pipeline stops. Did you get a fresh copy of the repository?
e-tomato
commented
1 year ago Hi @SvitlanaLukicheva,
It has been fixed now and MINI-EX works well. Thank you very much for your help.
Best,
SvitlanaLukicheva
commented
1 year ago Happy to hear that!
Dear MINI-EX developers @SvitlanaLukicheva @gabimadrid,
I encountered a new issue, as shown below in the file .nextflow.log. The last error message have been repeating for several hours and it seems like it will be runing endlessly. I have also tried removing the parallel analysis, but the problem is still there. How can I solve this? Thanks. Best,
.nextflow.log:
runing command:
miniex_run_slurm.config:
runing output