search

epi2me-labs / wf-bacterial-genomes

Small variant calling for haploid samples
https://labs.epi2me.io/
Other
26 stars 8 forks source link

[Bug]: error executing process 'calling_pipeline:run_isolates:processResfinder' #13

Closed samuelmontgomery closed 1 year ago

samuelmontgomery commented 1 year ago

What happened?

Hi, I am getting a consisent error when including --isolate true in my pipeline It appears the pipeline isn't writing the results of PointFinder to the work directory, as the pipeline is unable to find it to run 'calling_pipeline:run_isolates:processResfinder'

When navigating to the work directory, I can see that the file "PointFinder_results.txt" file does not exists, but the ResFinder results are there

Operating System

Windows 10

Workflow Execution

EPI2ME Labs desktop application

Workflow Execution - EPI2ME Labs Versions

No response

Workflow Execution - CLI Execution Profile

None

Workflow Version

v0.3

Relevant log output

Jun-19 04:49:42.954 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-19 04:49:42.954 [Task submitter] INFO  nextflow.Session - [a4/411820] Submitted process > calling_pipeline:run_isolates:resfinder (1)
Jun-19 04:50:41.769 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 23; name: calling_pipeline:run_isolates:resfinder (1); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/scratch/raw_data/nanopore/sa19/work/a4/4118209cff8e009ecdd2f27175e9c1]
Jun-19 04:50:41.780 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-19 04:50:41.780 [Task submitter] INFO  nextflow.Session - [19/c71662] Submitted process > calling_pipeline:run_isolates:processResfinder (1)
Jun-19 04:50:41.800 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jun-19 04:50:41.800 [Task submitter] INFO  nextflow.Session - [45/602b1a] Submitted process > calling_pipeline:makePerSampleReports (1)
Jun-19 04:51:06.198 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 25; name: calling_pipeline:makePerSampleReports (1); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/scratch/raw_data/nanopore/sa19/work/45/602b1a068f4d425447a8511e7acb53]
Jun-19 04:51:07.416 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 24; name: calling_pipeline:run_isolates:processResfinder (1); status: COMPLETED; exit: 1; error: -; workDir: /home/ubuntu/scratch/raw_data/nanopore/sa19/work/19/c71662a3a4cdbd3afaaed301540476]
Jun-19 04:51:07.419 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=calling_pipeline:run_isolates:processResfinder (1); work-dir=/home/ubuntu/scratch/raw_data/nanopore/sa19/work/19/c71662a3a4cdbd3afaaed301540476
  error [nextflow.exception.ProcessFailedException]: Process `calling_pipeline:run_isolates:processResfinder (1)` terminated with an error exit status (1)
Jun-19 04:51:07.427 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'calling_pipeline:run_isolates:processResfinder (1)'

Caused by:
  Process `calling_pipeline:run_isolates:processResfinder (1)` terminated with an error exit status (1)

Command executed:

  workflow-glue process_resfinder             --resfinder_file sa19_filtered_resfinder_results/ResFinder_results_tab.txt             --pointfinder_file sa19_filtered_resfinder_results/PointFinder_results.txt             --output sa19_filtered.resfinder_results.txt             --database_location sa19_filtered_resfinder_results/pointfinder_blast/tmp/

Command exit status:
  1

Command output:
  (empty)

Command error:
  [04:50:58 - workflow_glue] Starting entrypoint.
  Traceback (most recent call last):
    File "/home/ubuntu/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/bin/workflow-glue", line 7, in <module>
      cli()
    File "/home/ubuntu/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/bin/workflow_glue/__init__.py", line 62, in cli
      args.func(args)
    File "/home/ubuntu/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/bin/workflow_glue/process_resfinder.py", line 161, in main
      pointfinder_data = pd.read_csv(args.pointfinder_file, sep="\t")
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv
      return _read(filepath_or_buffer, kwds)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 577, in _read
      parser = TextFileReader(filepath_or_buffer, **kwds)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__
      self._engine = self._make_engine(f, self.engine)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1661, in _make_engine
      self.handles = get_handle(
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/io/common.py", line 859, in get_handle
      handle = open(
  FileNotFoundError: [Errno 2] No such file or directory: 'sa19_filtered_resfinder_results/PointFinder_results.txt'

Work dir:
  /home/ubuntu/scratch/raw_data/nanopore/sa19/work/19/c71662a3a4cdbd3afaaed301540476

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
Jun-19 04:51:07.429 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `calling_pipeline:run_isolates:processResfinder (1)` terminated with an error exit status (1)
Jun-19 04:51:07.442 [Task monitor] DEBUG nextflow.Session - The following nodes are still active:
[process] calling_pipeline:makeReport
  status=ACTIVE
  port 0: (value) bound ; channel: versions/*
  port 1: (value) bound ; channel: params.json
  port 2: (value) bound ; channel: variants/*
  port 3: (value) bound ; channel: sample_ids
  port 4: (value) bound ; channel: prokka/*
  port 5: (queue) closed; channel: per_read_stats
  port 6: (value) bound ; channel: fwd/*
  port 7: (value) bound ; channel: rev/*
  port 8: (value) bound ; channel: total_depth/*
  port 9: (value) bound ; channel: flye_stats/*
  port 10: (value) bound ; channel: resfinder/*
  port 11: (value) bound ; channel: mlst/*
  port 12: (cntrl) -     ; channel: $

[process] output
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: fname
  port 1: (cntrl) -     ; channel: $

Jun-19 04:51:07.454 [Actor Thread 13] DEBUG nextflow.util.CacheHelper - Hash asset file sha-256: /home/ubuntu/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/data/OPTIONAL_FILE
Jun-19 04:51:07.457 [Task submitter] DEBUG n.processor.TaskPollingMonitor - %% executor local > tasks in the submission queue: 1 -- tasks to be submitted are shown below
~> TaskHandler[id: 26; name: calling_pipeline:makeReport (1); status: NEW; exit: -; error: -; workDir: /home/ubuntu/scratch/raw_data/nanopore/sa19/work/0e/1029af066443477807ff5d62f0630e]
Jun-19 04:51:08.400 [main] DEBUG nextflow.Session - Session await > all processes finished
Jun-19 04:51:08.400 [main] DEBUG nextflow.Session - Session await > all barriers passed
Jun-19 04:51:08.637 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=24; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=1; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=55m 13s; failedDuration=25.5s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=4; peakCpus=4; peakMemory=0; ]
Jun-19 04:51:08.637 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Jun-19 04:51:08.638 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
cjalder commented 1 year ago

Hi Samuel, Thanks for contacting us about this issue. You're right it does look like it's trying to process the Pointfinder results when they don't exist. Can I ask if there is a JSON file within the results finder output directory? It might help us with determining the root of the error.

samuelmontgomery commented 1 year ago

There is no JSON file in the work directory or the output directory for the pipeline

In the workdirectory it has the resfinder & disinfinder results, but no output for pointfinder It does have a temporary directory called pointfinder_blast with .xml files

cjalder commented 1 year ago

Ok thanks for checking. Are you able to go the working directory of the mlstSearch process to see what the output from that is?

samuelmontgomery commented 1 year ago

I get "input_genome.fasta" and "sa19_filtered.mlst.json"

cjalder commented 1 year ago

Could you check the contents of "sa19_filtered.mlst.json" and let us know the result, specifically on the field "scheme"?

cjalder commented 1 year ago

Hi,

The latest version of the workflow v0.3.1 has fixes for somes bugs that we had noticed in AMR SNP calling, particularly with S.aureus, please let us know if this fixes your issue

mattdmem commented 1 year ago

@samuelmontgomery - we'll close this for now. Please do reach out if you are still experiencing issues with AMR calling in this workflow.