search

epi2me-labs / wf-pore-c

Other
29 stars 8 forks source link

error [nextflow.exception.ProcessFailedException]: Process `POREC:merge_pairs (1)` terminated with an error exit status (1) #70

Open wangguiqian opened 1 month ago

wangguiqian commented 1 month ago

Operating System

CentOS 7

Other Linux

CentOS 7

Workflow Version

V1.2.1

Workflow Execution

EPI2ME Desktop (Local)

Other workflow execution

nextflow run ~/Software/wf-pore-c-1.1.0/ --fastq ../raw_data/Pore-C_part1.fq.gz --ref ../Gh_genome.fa --cutter DpnII -c increase_memory.config --threads 35 --paired_end_minimum_distance 100 --paired_end_maximum_distance 200 --chunk_size 10000 --filter_pairs --paired_end --hi_c --out_dir PoreC_part1 -profile singularity -with-singularity '~/Software/wf-pore-c-1.1.0/ontresearch-wf-pore-c-sha3787c234c0cacf66a67fb77da223cc2e1cb0baf0.sif' -with-trace -resume

EPI2ME Version

No response

CLI command run

This my offline to run command: nextflow run ~/Software/wf-pore-c-1.1.0/ --fastq ../raw_data/Pore-C_part1.fq.gz --ref ../Gh_genome.fa --cutter DpnII -c increase_memory.config --threads 35 --paired_end_minimum_distance 100 --paired_end_maximum_distance 200 --chunk_size 10000 --filter_pairs --paired_end --hi_c --out_dir PoreC_part1 -profile singularity -with-singularity '~/Software/wf-pore-c-1.1.0/ontresearch-wf-pore-c-sha3787c234c0cacf66a67fb77da223cc2e1cb0baf0.sif' -with-trace -resume

But my program ran in offline mode, and after about a week of running it, this error occurred.I didn’t to solve the problem ,why occurred the errors ? This last discussed with you the question https://github.com/epi2me-labs/wf-pore-c/issues/64#event-13534065805 Please to see the error message :

Aug-09 15:57:25.188 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 7231; name: POREC:merge_pairs_stats (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/ba/e6f19f5e350fd9bc1ac8cfd9add7d6] Aug-09 15:59:12.358 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 7230; name: POREC:merge_pairs (1); status: COMPLETED; exit: 1; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/8a/73727d503a81db04665630f9e64eff] Aug-09 15:59:12.363 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=POREC:merge_pairs (1); work-dir=/public/home/yqming/Work_plate/Cotton_data/Pore_data/work/8a/73727d503a81db04665630f9e64eff error [nextflow.exception.ProcessFailedException]: Process POREC:merge_pairs (1) terminated with an error exit status (1) Aug-09 15:59:12.488 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'POREC:merge_pairs (1)'

Caused by: Process POREC:merge_pairs (1) terminated with an error exit status (1)

Command executed:

pass a quoted glob, pairtools will do its own globbing

pairtools merge -o "null.pairs.gz" --concatenate 'to_merge/*'

Command exit status: 1

Command output: (empty)

Command error: Traceback (most recent call last): File "/home/epi2melabs/conda/bin/pairtools", line 11, in sys.exit(cli()) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/home/epi2melabs/conda/lib/python3.8/site-packages/pairtools/cli/merge.py", line 134, in merge merge_py( File "/home/epi2melabs/conda/lib/python3.8/site-packages/pairtools/cli/merge.py", line 254, in merge_py subprocess.check_call(command, shell=True, stdout=outstream) File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 359, in check_call retcode = call(*popenargs, *kwargs) File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 340, in call with Popen(popenargs, kwargs) as p: File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 858, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) OSError: [Errno 7] Argument list too long: '/bin/sh'

Work dir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/8a/73727d503a81db04665630f9e64eff

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run Aug-09 15:59:12.509 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process POREC:merge_pairs (1) terminated with an error exit status (1) Aug-09 15:59:12.550 [Task monitor] DEBUG nextflow.Session - The following nodes are still active: [process] POREC:pair_stats_report status=ACTIVE port 0: (queue) OPEN ; channel: - port 1: (cntrl) - ; channel: $

Aug-09 15:59:12.633 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Aug-09 15:59:12.633 [Task submitter] INFO nextflow.Session - [ba/e6f19f] Submitted process > POREC:merge_pairs_stats (1) Aug-09 15:59:12.712 [main] DEBUG nextflow.Session - Session await > all processes finished Aug-09 15:59:12.712 [main] DEBUG nextflow.Session - Session await > all barriers passed Aug-09 15:59:12.728 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop Aug-09 15:59:12.745 [main] WARN n.processor.TaskPollingMonitor - Killing running tasks (1) Aug-09 15:59:12.772 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=7229; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=1; runningCount=-1; retriesCount=0; abortedCount=1; succeedDuration=212d 2h 32m 23s; failedDuration=1h 55m 5s; cachedDuration=0ms;loadCpus=-30; loadMemory=0; peakRunning=2; peakCpus=35; peakMemory=122 GB; ] Aug-09 15:59:12.773 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file Aug-09 15:59:12.842 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report Aug-09 15:59:15.271 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline Aug-09 15:59:15.761 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done Aug-09 15:59:15.802 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Workflow Execution - CLI Execution Profile

None

What happened?

Aug-09 15:57:25.188 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 7231; name: POREC:merge_pairs_stats (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/ba/e6f19f5e350fd9bc1ac8cfd9add7d6] Aug-09 15:59:12.358 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 7230; name: POREC:merge_pairs (1); status: COMPLETED; exit: 1; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/8a/73727d503a81db04665630f9e64eff] Aug-09 15:59:12.363 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=POREC:merge_pairs (1); work-dir=/public/home/yqming/Work_plate/Cotton_data/Pore_data/work/8a/73727d503a81db04665630f9e64eff error [nextflow.exception.ProcessFailedException]: Process POREC:merge_pairs (1) terminated with an error exit status (1) Aug-09 15:59:12.488 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'POREC:merge_pairs (1)'

Caused by: Process POREC:merge_pairs (1) terminated with an error exit status (1)

Command executed:

pass a quoted glob, pairtools will do its own globbing

pairtools merge -o "null.pairs.gz" --concatenate 'to_merge/*'

Command exit status: 1

Command output: (empty)

Command error: Traceback (most recent call last): File "/home/epi2melabs/conda/bin/pairtools", line 11, in sys.exit(cli()) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/home/epi2melabs/conda/lib/python3.8/site-packages/pairtools/cli/merge.py", line 134, in merge merge_py( File "/home/epi2melabs/conda/lib/python3.8/site-packages/pairtools/cli/merge.py", line 254, in merge_py subprocess.check_call(command, shell=True, stdout=outstream) File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 359, in check_call retcode = call(*popenargs, *kwargs) File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 340, in call with Popen(popenargs, kwargs) as p: File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 858, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) OSError: [Errno 7] Argument list too long: '/bin/sh'

Work dir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/8a/73727d503a81db04665630f9e64eff

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run Aug-09 15:59:12.509 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process POREC:merge_pairs (1) terminated with an error exit status (1) Aug-09 15:59:12.550 [Task monitor] DEBUG nextflow.Session - The following nodes are still active: [process] POREC:pair_stats_report status=ACTIVE port 0: (queue) OPEN ; channel: - port 1: (cntrl) - ; channel: $

Aug-09 15:59:12.633 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Aug-09 15:59:12.633 [Task submitter] INFO nextflow.Session - [ba/e6f19f] Submitted process > POREC:merge_pairs_stats (1) Aug-09 15:59:12.712 [main] DEBUG nextflow.Session - Session await > all processes finished Aug-09 15:59:12.712 [main] DEBUG nextflow.Session - Session await > all barriers passed Aug-09 15:59:12.728 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop Aug-09 15:59:12.745 [main] WARN n.processor.TaskPollingMonitor - Killing running tasks (1) Aug-09 15:59:12.772 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=7229; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=1; runningCount=-1; retriesCount=0; abortedCount=1; succeedDuration=212d 2h 32m 23s; failedDuration=1h 55m 5s; cachedDuration=0ms;loadCpus=-30; loadMemory=0; peakRunning=2; peakCpus=35; peakMemory=122 GB; ] Aug-09 15:59:12.773 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file Aug-09 15:59:12.842 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report Aug-09 15:59:15.271 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline Aug-09 15:59:15.761 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done Aug-09 15:59:15.802 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Relevant log output

Aug-02 17:13:49.033 [main] DEBUG nextflow.cli.Launcher - $> nextflow run /public/home/yqming/Software/wf-pore-c-1.1.0/ --fastq ../raw_data/Pore-C_part1.fq.gz --ref ../Gh_genome.fa --cutter DpnII -c increase_memory.config --threads 35 --paired_end_minimum_distance 100 --paired_end_maximum_distance 200 --chunk_size 10000 --filter_pairs --paired_end --hi_c --out_dir PoreC_part1 -profile singularity -with-singularity ~/Software/wf-pore-c-1.1.0/ontresearch-wf-pore-c-sha3787c234c0cacf66a67fb77da223cc2e1cb0baf0.sif -with-trace -resume
Aug-02 17:13:49.127 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.10.1
Aug-02 17:13:49.162 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/public/home/yqming/.nextflow/plugins; core-plugins: nf-amazon@2.1.4,nf-azure@1.3.3,nf-cloudcache@0.3.0,nf-codecommit@0.1.5,nf-console@1.0.6,nf-ga4gh@1.1.0,nf-google@1.8.3,nf-tower@1.6.3,nf-wave@1.0.1
Aug-02 17:13:49.180 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Aug-02 17:13:49.182 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Aug-02 17:13:49.187 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Aug-02 17:13:49.203 [main] INFO  org.pf4j.AbstractPluginManager - No plugins

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

sarahjeeeze commented 4 weeks ago

thanks for reporting, we will look in to this a get back to you. How large in GB was your input dataset? In the meantime i would suggest running again with larger chunk size - perhaps the default 25000

wangguiqian commented 4 weeks ago

thanks for reporting, we will look in to this a get back to you. How large in GB was your input dataset? In the meantime i would suggest running again with larger chunk size - perhaps the default 25000

My dataset is 80GB ,I seting 35 threads to run .Thanks you help me

sarahjeeeze commented 3 weeks ago

Hey, are you definitely running the latest versoin - i notice it says Software/wf-pore-c-1.1.0 - and in that version a bug that caused this error had not yet been fixed.

wangguiqian commented 3 weeks ago

Hey, are you definitely running the latest versoin - i notice it says Software/wf-pore-c-1.1.0 - and in that version a bug that caused this error had not yet been fixed.

yes ,I used the version wf-pore-c-1.1.0 .Do you know what caused this error ?

wangguiqian commented 2 weeks ago

I followed your suggestion to modify the --chunk_size parameter of the run command again, but when I ran it again, the bug appeared again and the run was automatically interrupted on the 7th day. I don't know why this happened. Is there any other way to run it? My fastq data is about 80GB. How can I run it in a distributed manner?

Below is the command I ran and the information I just reported： nextflow run /public/home/yqming/Software/wf-pore-c-1.1.0/ --fastq ../raw_data/Pore-C_part1.fq.gz --ref ../Gh_genome.fa --cutter DpnII -c increase_memory.config --threads 35 --paired_end_minimum_distance 100 --paired_end_maximum_distance 200 --chunk_size 40000 --filter_pairs --paired_end --hi_c --out_dir PoreC_part1 -profile singularity -with-singularity ~/Software/wf-pore-c-1.1.0/ontresearch-wf-pore-c-sha3787c234c0cacf66a67fb77da223cc2e1cb0baf0.sif -with-trace -resume Aug-16 22:08:20.349 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 23.10.1 Aug-16 22:08:20.382 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/public/home/yqming/.nextflow/plugins; core-plugins: nf-amazon@2.1.4,nf-azure@1.3.3,nf-cloudcache@0.3.0,nf-codecommit@0.1.5,nf-console@1.0.6,nf-ga4gh@1.1.0,nf-google@1.8.3,nf-tower@1.6.3,nf-wave@1.0.1 Aug-16 22:08:20.410 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: [] Aug-16 22:08:20.412 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: [] Aug-16 22:08:20.417 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode Aug-16 22:08:20.430 [main] INFO org.pf4j.AbstractPluginManager - No plugins Aug-16 22:08:21.794 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /public/home/yqming/Software/wf-pore-c-1.1.0/nextflow.config Aug-16 22:08:21.815 [main] DEBUG nextflow.config.ConfigBuilder - User config file: /public/home/yqming/Work_plate/Cotton_data/Pore_data/increase_memory.config Aug-16 22:08:21.816 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /public/home/yqming/Software/wf-pore-c-1.1.0/nextflow.config Aug-16 22:08:21.816 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /public/home/yqming/Work_plate/Cotton_data/Pore_data/increase_memory.config Aug-16 22:08:21.835 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: singularity Aug-16 22:08:22.041 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: singularity Aug-16 22:08:22.070 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [standard, conda, singularity, awsbatch, local] Aug-16 22:08:22.081 [main] WARN nextflow.config.ConfigBuilder - It appears you have never run this project before -- Option -resume is ignored Aug-16 22:08:22.108 [main] DEBUG nextflow.config.ConfigBuilder - Enabling execution in Singularity container as requested by command-line option -with-singularity null Aug-16 22:08:22.144 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion Aug-16 22:08:22.179 [main] INFO nextflow.cli.CmdRun - Launching /public/home/yqming/Software/wf-pore-c-1.1.0/main.nf [mad_elion] DSL2 - revision: cc183d633a Aug-16 22:08:22.180 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[] Aug-16 22:08:22.180 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[] Aug-16 22:08:22.194 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /public/home/yqming/.nextflow/secrets/store.json Aug-16 22:08:22.199 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@67332b1e] - activable => nextflow.secret.LocalSecretsProvider@67332b1e Aug-16 22:08:22.295 [main] DEBUG nextflow.Session - Session UUID: a11272e8-7185-4501-81d4-75efc2655b24 Aug-16 22:08:22.296 [main] DEBUG nextflow.Session - Run name: mad_elion Aug-16 22:08:22.297 [main] DEBUG nextflow.Session - Executor pool size: 36 Aug-16 22:08:22.309 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null Aug-16 22:08:22.315 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=108; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false Aug-16 22:08:22.398 [main] DEBUG nextflow.cli.CmdRun - Version: 23.10.1 build 5891 Created: 12-01-2024 22:01 UTC (13-01-2024 06:01 CDT) System: Linux 3.10.0-862.el7.x86_64 Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 11.0.13+7-b1751.21 Encoding: UTF-8 (UTF-8) Process: 275840@c05n10 [10.1.5.10] CPUs: 36 - Mem: 180 GB (179.5 GB) - Swap: 0 (0) Aug-16 22:08:22.440 [main] DEBUG nextflow.Session - Work-dir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work [gpfs] Aug-16 22:08:22.511 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[] Aug-16 22:08:22.525 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory Aug-16 22:08:22.580 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory Aug-16 22:08:22.597 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 37; maxThreads: 1000 Aug-16 22:08:22.770 [main] DEBUG nextflow.Session - Session start Aug-16 22:08:22.779 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /public/home/yqming/Work_plate/Cotton_data/Pore_data/PoreC_part1/execution/trace.txt Aug-16 22:08:22.800 [main] DEBUG nextflow.Session - Using default localLib path: /public/home/yqming/Software/wf-pore-c-1.1.0/lib Aug-16 22:08:22.863 [main] DEBUG nextflow.Session - Adding to the classpath library: /public/home/yqming/Software/wf-pore-c-1.1.0/lib Aug-16 22:08:22.864 [main] DEBUG nextflow.Session - Adding to the classpath library: /public/home/yqming/Software/wf-pore-c-1.1.0/lib/nfcore_external_javadeps.jar Aug-16 22:08:25.385 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution Aug-16 22:08:27.387 [main] INFO nextflow.Nextflow - |||||||||| ____ _ __ _ |||||||||| | _| _ | | \/ | __| | | | | ||||| | | | |) | | ) | |\/| | _| ___| |/ ` | ' \/ | ||||| | |_| /| | / /| | | | |_|| | (| | |) _ \ |||||||||| |____|_| |_|___|| ||| ||\,|._/|/ |||||||||| wf-pore-c v1.1.0 -------------------------------------------------------------------------------- Core Nextflow options runName : mad_elion containerEngine : singularity container : [withLabel:wfporec:/public/home/yqming/Software/wf-pore-c-1.1.0/ontresearch-wf-pore-c-sha3787c234c0cacf66a67fb77da223cc2e1cb0baf0.sif, withLabel:pore_c_py:/public/home/yqming/Software/wf-pore-c-1.1.0/ontresearch-pore-c-py-shad5608368dcce93646de5559f2f391005408da6c2.sif, withLabel:wf_common:/public/home/yqming/Software/wf-pore-c-1.1.0/wf-common_sha1c5febff9f75143710826498b093d9769a5edbb9.sif, default:~/Software/wf-pore-c-1.1.0/ontresearch-wf-pore-c-sha3787c234c0cacf66a67fb77da223cc2e1cb0baf0.sif] launchDir : /public/home/yqming/Work_plate/Cotton_data/Pore_data workDir : /public/home/yqming/Work_plate/Cotton_data/Pore_data/work projectDir : /public/home/yqming/Software/wf-pore-c-1.1.0 userName : yqming profile : singularity configFiles : /public/home/yqming/Software/wf-pore-c-1.1.0/nextflow.config, /public/home/yqming/Work_plate/Cotton_data/Pore_data/increase_memory.config

Input Options fastq : ../raw_data/Pore-C_part1.fq.gz ref : ../Gh_genome.fa cutter : DpnII

Output Options out_dir : PoreC_part1 hi_c : true

Advanced Options chunk_size : 40000 threads : 35

Paired-end BAM Options paired_end : true filter_pairs : true paired_end_minimum_distance: 100 paired_end_maximum_distance: 200

!! Only displaying parameters that differ from the pipeline defaults !! -------------------------------------------------------------------------------- If you use epi2me-labs/wf-pore-c for your analysis please cite:

_This is the error message:_ Aug-20 12:14:22.681 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 1821; name: POREC:pair_stats_report (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/a5/65a3c8c05975fdef858bed2bc02278] Aug-20 12:19:21.607 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below ~> TaskHandler[id: 1820; name: POREC:prepare_hic (1); status: RUNNING; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6] Aug-20 12:19:22.732 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 1821; name: POREC:pair_stats_report (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/a5/65a3c8c05975fdef858bed2bc02278] Aug-20 12:24:21.670 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below ~> TaskHandler[id: 1820; name: POREC:prepare_hic (1); status: RUNNING; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6] Aug-20 12:24:22.785 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 1821; name: POREC:pair_stats_report (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/a5/65a3c8c05975fdef858bed2bc02278] Aug-20 12:29:21.685 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below ~> TaskHandler[id: 1820; name: POREC:prepare_hic (1); status: RUNNING; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6] Aug-20 12:29:22.842 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 1821; name: POREC:pair_stats_report (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/a5/65a3c8c05975fdef858bed2bc02278] Aug-20 12:34:21.784 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below ~> TaskHandler[id: 1820; name: POREC:prepare_hic (1); status: RUNNING; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6] Aug-20 12:34:22.916 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 1821; name: POREC:pair_stats_report (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/a5/65a3c8c05975fdef858bed2bc02278] Aug-20 12:39:21.850 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below ~> TaskHandler[id: 1820; name: POREC:prepare_hic (1); status: RUNNING; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6] Aug-20 12:39:22.961 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 1821; name: POREC:pair_stats_report (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/a5/65a3c8c05975fdef858bed2bc02278] Aug-20 12:44:21.892 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below ~> TaskHandler[id: 1820; name: POREC:prepare_hic (1); status: RUNNING; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6] Aug-20 12:44:23.010 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 1821; name: POREC:pair_stats_report (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/a5/65a3c8c05975fdef858bed2bc02278] Aug-20 12:49:21.971 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 1 -- submitted tasks are shown below ~> TaskHandler[id: 1820; name: POREC:prepare_hic (1); status: RUNNING; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6] Aug-20 12:49:23.164 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below ~> TaskHandler[id: 1821; name: POREC:pair_stats_report (1); status: NEW; exit: -; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/a5/65a3c8c05975fdef858bed2bc02278] Aug-20 12:49:37.792 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1820; name: POREC:prepare_hic (1); status: COMPLETED; exit: 127; error: -; workDir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6] Aug-20 12:49:37.837 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Aug-20 12:49:37.837 [Task submitter] INFO nextflow.Session - [a5/65a3c8] Submitted process > POREC:pair_stats_report (1) Aug-20 12:49:37.841 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=POREC:prepare_hic (1); work-dir=/public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6 error [nextflow.exception.ProcessFailedException]: Process POREC:prepare_hic (1) terminated with an error exit status (127) Aug-20 12:49:38.100 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'POREC:prepare_hic (1)'

Caused by: Process POREC:prepare_hic (1) terminated with an error exit status (127)

Command executed:

cut -f1,2 fasta.fai > sizes.genome pairtools flip input.pairs.gz -c sizes.genome > flipped.pairs.tmp pairtools sort flipped.pairs.tmp > sorted.pairs.tmp pairtools dedup --chunksize 100000 sorted.pairs.tmp > dedup.pairs.tmp java -jar /home/epi2melabs/juicer_tools_1.22.01.jar pre dedup.pairs.tmp "null.hic" sizes.genome rm -rf "*.pairs.tmp"

Command exit status: 127

Command output: (empty)

Command error: .command.sh: line 6: java: command not found

Work dir: /public/home/yqming/Work_plate/Cotton_data/Pore_data/work/b3/cbc440b8048ee9a1097a1867d67ce6

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out Aug-20 12:49:38.121 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process POREC:prepare_hic (1) terminated with an error exit status (127) Aug-20 12:49:38.269 [main] DEBUG nextflow.Session - Session await > all processes finished Aug-20 12:49:38.269 [main] DEBUG nextflow.Session - Session await > all barriers passed Aug-20 12:49:38.283 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop Aug-20 12:49:38.301 [main] WARN n.processor.TaskPollingMonitor - Killing running tasks (1) Aug-20 12:49:38.318 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=1819; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=1; runningCount=-1; retriesCount=0; abortedCount=1; succeedDuration=108d 3h 59m 45s; failedDuration=2d 5h 24m 16s; cachedDuration=0ms;loadCpus=-30; loadMemory=0; peakRunning=2; peakCpus=35; peakMemory=122 GB; ] Aug-20 12:49:38.319 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file Aug-20 12:49:38.410 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report Aug-20 12:49:39.809 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline Aug-20 12:49:40.126 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done Aug-20 12:49:40.150 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

wangguiqian commented 2 weeks ago

wang.txt this my log file, Please help for me to check .