search

epi2me-labs / wf-artic

ARTIC SARS-CoV-2 workflow and reporting
https://labs.epi2me.io/
Other
47 stars 34 forks source link

[Bug]: sed: couldn't open temporary file ./sed3unqLx: Permission denied #70

Closed corneliusroemer closed 1 year ago

corneliusroemer commented 1 year ago

What happened?

A bug happened!

I ran pretty standard wf-artic on a single fastq in my downloads folder and got a sed error: sed: couldn't open temporary file ./sed3unqLx: Permission denied

This is what I ran: nextflow run epi2me-labs/wf-artic -r v0.3.18 --fastq ~/Downloads/barcode95.fastq.gz --scheme_version Midnight-ONT/V3

This looks like some permissions thing on mac, I'm notably running the pipeline in my Downloads folder. But things went smoothly up to this point. So something is strange.

I'm not sure how I'm running it - docker or local? I'm just executing in the terminal - so conda???

Also not quite sure why I get multiple times quite similar output. I don't have any nextflow experience so sorry for my noob questions.

Operating System

macOS

Workflow Execution

Command line

Workflow Execution - EPI2ME Labs Versions

NA

Workflow Execution - Execution Profile

container Engine docker

Workflow Version

0.3.18

Relevant log output

❯ nextflow run epi2me-labs/wf-artic -r v0.3.18 --fastq  ~/Downloads/barcode95.fastq.gz  --scheme_version Midnight-ONT/V3 
N E X T F L O W  ~  version 22.10.1
Launching `https://github.com/epi2me-labs/wf-artic` [nostalgic_sax] DSL2 - revision: a60a1e1e73 [v0.3.18]

WARN: Found unexpected parameters:
* --scheme_dir: primer_schemes
- Ignore this warning: params.schema_ignore_params = "scheme_dir" 

Core Nextflow options
  revision       : v0.3.18
  runName        : nostalgic_sax
  containerEngine: docker
  launchDir      : /Users/corneliusromer/Downloads/BJGMC_ONT_Run71
  workDir        : /Users/corneliusromer/Downloads/BJGMC_ONT_Run71/work
  projectDir     : /Users/corneliusromer/.nextflow/assets/epi2me-labs/wf-artic
  userName       : corneliusromer
  profile        : standard
  configFiles    : /Users/corneliusromer/.nextflow/assets/epi2me-labs/wf-artic/nextflow.config

Basic Input/Output Options
  fastq          : /Users/corneliusromer/Downloads/barcode95.fastq.gz

Primer Scheme Selection
  scheme_version : Midnight-ONT/V3

Advanced options
  normalise      : 200

Checking fastq input.
Single file input detected.
executor >  local (7)
[78/fdf9e9] process > handleSingleFile (1)    [100%] 1 of 1 ✔
[59/1ed9df] process > pipeline:getVersions    [100%] 1 of 1 ✔
[b9/488a20] process > pipeline:getParams      [100%] 1 of 1 ✔
[2a/d2faa0] process > pipeline:copySchemeDir  [100%] 1 of 1 ✔
[d1/9cf105] process > pipeline:preArticQC (1) [100%] 1 of 1 ✔
[48/570c50] process > pipeline:runArtic (1)   [  0%] 0 of 1
[-        ] process > pipeline:combineDepth   -
[-        ] process > pipeline:allConsensus   -
[-        ] process > pipeline:allVariants    -
[81/20326a] process > pipeline:prep_nextclade [100%] 1 of 1 ✔
[-        ] process > pipeline:nextclade      -
[-        ] process > pipeline:pangolin       -
[-        ] process > pipeline:telemetry      -
[-        ] process > pipeline:report         -
[-        ] process > output                  -
Error executing process > 'pipeline:runArtic (1)'

Caused by:
  Process `pipeline:runArtic (1)` terminated with an error exit status (4)

Command executed:

  run_artic.sh         barcode95 barcode95 150 1200         r941_min_hac_variant_g507 SARS-CoV-2 primer_schemes         Midnight-ONT/V3 2 0 200
  bcftools stats barcode95.pass.named.vcf.gz > barcode95.pass.named.stats

Command exit status:
  4

Command output:
  barcode95_barcode95.fastq     13152
   - artic guppyplex finished
  Running hacked up minion.py

Command error:
  [00:09:34 - TrimOlap] MN908947.3:1885.6-2261.0 and MN908947.3:3144.0-3572.2 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:35 - TrimOlap] MN908947.3:3907.42-4249.0 and MN908947.3:7298.0-7646.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:35 - TrimOlap] MN908947.3:8004.1-8384.0 and MN908947.3:9317.0-10095.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:36 - TrimOlap] MN908947.3:9653.0-10429.0 and MN908947.3:11407.0-12165.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:11705.0-12459.0 and MN908947.3:14331.0-14581.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:14331.0-14581.0 and MN908947.3:15617.0-16226.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:16241.0-16708.0 and MN908947.3:17634.0-18170.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:18088.0-18691.0 and MN908947.3:19588.0-20318.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:19839.0-20587.0 and MN908947.3:21641.0-21941.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:21641.0-21941.0 and MN908947.3:22090.0-22593.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:22090.0-22593.0 and MN908947.3:23554.0-23999.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:39 - TrimOlap] MN908947.3:23554.0-23999.0 and MN908947.3:25708.0-26428.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:48 - ValidArgs] Reads will be filtered to only those with RG tag: 2
  [00:09:48 - Annotate] Getting chrom coordinates
  [00:09:53 - Annotate] Processing chunk with coordinates: MN908947.3:1113-26810
  Lines   total/split/realigned/skipped:        4/0/0/0
  Note: the --sample option not given, applying all records regardless of the genotype
  Applied 4 variants
  Running: minimap2 -a -x map-ont -t 2 primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95_barcode95.fastq | samtools view -bS -F 4 - | samtools sort -o barcode95.sorted.bam -
  Running: samtools index barcode95.sorted.bam
  Running: align_trim barcode95.sorted.bam primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.scheme.bed  --normalise 200 --start --report barcode95.alignreport.txt 2> barcode95.alignreport.er | samtools sort -T barcode95 - -o barcode95.trimmed.rg.sorted.bam
  Running: align_trim barcode95.sorted.bam primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.scheme.bed  --normalise 200 --report barcode95.alignreport.txt 2> barcode95.alignreport.er | samtools sort -T barcode95 - -o barcode95.primertrimmed.rg.sorted.bam
  Running: samtools index barcode95.trimmed.rg.sorted.bam
  Running: samtools index barcode95.primertrimmed.rg.sorted.bam
  Running: medaka consensus --model r941_min_hac_variant_g507 --threads 2 --chunk_len 800 --chunk_ovlp 400 --RG 1 barcode95.trimmed.rg.sorted.bam barcode95.1.hdf
  Running: medaka variant --gvcf primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95.1.hdf barcode95.1.vcf
executor >  local (7)[78/fdf9e9] process > handleSingleFile (1)    [100%] 1 of 1 ✔
[59/1ed9df] process > pipeline:getVersions    [100%] 1 of 1 ✔
[b9/488a20] process > pipeline:getParams      [100%] 1 of 1 ✔[2a/d2faa0] process > pipeline:copySchemeDir  [100%] 1 of 1 ✔
[d1/9cf105] process > pipeline:preArticQC (1) [100%] 1 of 1 ✔
[48/570c50] process > pipeline:runArtic (1)   [100%] 1 of 1, failed: 1 ✘[-        ] process > pipeline:combineDepth   -
[-        ] process > pipeline:allConsensus   -
[-        ] process > pipeline:allVariants    [  0%] 0 of 1[81/20326a] process > pipeline:prep_nextclade [100%] 1 of 1 ✔
[-        ] process > pipeline:nextclade      -
[-        ] process > pipeline:pangolin       -
[-        ] process > pipeline:telemetry      -
[-        ] process > pipeline:report         -
[-        ] process > output                  -
WARN: Input tuple does not match input set cardinality declared by process `pipeline:telemetry` -- offending value: [[]]
WARN: Input tuple does not match input set cardinality declared by process `pipeline:allVariants` -- offending value: [[]]
Error executing process > 'pipeline:runArtic (1)'

Caused by:  Process `pipeline:runArtic (1)` terminated with an error exit status (4)
Command executed:
  run_artic.sh         barcode95 barcode95 150 1200         r941_min_hac_variant_g507 SARS-CoV-2 primer_schemes         Midnight-ONT/V3 2 0 200
  bcftools stats barcode95.pass.named.vcf.gz > barcode95.pass.named.stats
Command exit status:
  4

Command output:
  barcode95_barcode95.fastq     13152
   - artic guppyplex finished
  Running hacked up minion.py

Command error:
  [00:09:34 - TrimOlap] MN908947.3:1885.6-2261.0 and MN908947.3:3144.0-3572.2 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:35 - TrimOlap] MN908947.3:3907.42-4249.0 and MN908947.3:7298.0-7646.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:35 - TrimOlap] MN908947.3:8004.1-8384.0 and MN908947.3:9317.0-10095.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:36 - TrimOlap] MN908947.3:9653.0-10429.0 and MN908947.3:11407.0-12165.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:11705.0-12459.0 and MN908947.3:14331.0-14581.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:14331.0-14581.0 and MN908947.3:15617.0-16226.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:16241.0-16708.0 and MN908947.3:17634.0-18170.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:18088.0-18691.0 and MN908947.3:19588.0-20318.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:19839.0-20587.0 and MN908947.3:21641.0-21941.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:21641.0-21941.0 and MN908947.3:22090.0-22593.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:22090.0-22593.0 and MN908947.3:23554.0-23999.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:39 - TrimOlap] MN908947.3:23554.0-23999.0 and MN908947.3:25708.0-26428.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:48 - ValidArgs] Reads will be filtered to only those with RG tag: 2
  [00:09:48 - Annotate] Getting chrom coordinates
  [00:09:53 - Annotate] Processing chunk with coordinates: MN908947.3:1113-26810
  Lines   total/split/realigned/skipped:        4/0/0/0
  Note: the --sample option not given, applying all records regardless of the genotype
  Applied 4 variants
  Running: minimap2 -a -x map-ont -t 2 primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95_barcode95.fastq | samtools view -bS -F 4 - | samtools sort -o barcode95.sorted.bam -
  Running: samtools index barcode95.sorted.bam
  Running: align_trim barcode95.sorted.bam primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.scheme.bed  --normalise 200 --start --report barcode95.alignreport.txt 2> barcode95.alignreport.er | samtools sort -T barcode95 - -o barcode95.trimmed.rg.sorted.bam
  Running: align_trim barcode95.sorted.bam primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.scheme.bed  --normalise 200 --report barcode95.alignreport.txt 2> barcode95.alignreport.er | samtools sort -T barcode95 - -o barcode95.primertrimmed.rg.sorted.bam
  Running: samtools index barcode95.trimmed.rg.sorted.bam
  Running: samtools index barcode95.primertrimmed.rg.sorted.bam
  Running: medaka consensus --model r941_min_hac_variant_g507 --threads 2 --chunk_len 800 --chunk_ovlp 400 --RG 1 barcode95.trimmed.rg.sorted.bam barcode95.1.hdf
  Running: medaka variant --gvcf primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95.1.hdf barcode95.1.vcf
executor >  local (7)[78/fdf9e9] process > handleSingleFile (1)    [100%] 1 of 1 ✔
[59/1ed9df] process > pipeline:getVersions    [100%] 1 of 1 ✔
[b9/488a20] process > pipeline:getParams      [100%] 1 of 1 ✔[2a/d2faa0] process > pipeline:copySchemeDir  [100%] 1 of 1 ✔
[d1/9cf105] process > pipeline:preArticQC (1) [100%] 1 of 1 ✔
[48/570c50] process > pipeline:runArtic (1)   [100%] 1 of 1, failed: 1 ✘[-        ] process > pipeline:combineDepth   -
[-        ] process > pipeline:allConsensus   -
[-        ] process > pipeline:allVariants    [  0%] 0 of 1[81/20326a] process > pipeline:prep_nextclade [100%] 1 of 1 ✔
[-        ] process > pipeline:nextclade      -
[-        ] process > pipeline:pangolin       -
[-        ] process > pipeline:telemetry      [  0%] 0 of 1
[-        ] process > pipeline:report         -
[-        ] process > output                  -
WARN: Input tuple does not match input set cardinality declared by process `pipeline:telemetry` -- offending value: [[]]
WARN: Input tuple does not match input set cardinality declared by process `pipeline:allVariants` -- offending value: [[]]
Error executing process > 'pipeline:runArtic (1)'

Caused by:  Process `pipeline:runArtic (1)` terminated with an error exit status (4)
Command executed:
  run_artic.sh         barcode95 barcode95 150 1200         r941_min_hac_variant_g507 SARS-CoV-2 primer_schemes         Midnight-ONT/V3 2 0 200
  bcftools stats barcode95.pass.named.vcf.gz > barcode95.pass.named.stats
Command exit status:
  4

Command output:
  barcode95_barcode95.fastq     13152
   - artic guppyplex finished
  Running hacked up minion.py

Command error:
  [00:09:34 - TrimOlap] MN908947.3:1885.6-2261.0 and MN908947.3:3144.0-3572.2 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:35 - TrimOlap] MN908947.3:3907.42-4249.0 and MN908947.3:7298.0-7646.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:35 - TrimOlap] MN908947.3:8004.1-8384.0 and MN908947.3:9317.0-10095.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:36 - TrimOlap] MN908947.3:9653.0-10429.0 and MN908947.3:11407.0-12165.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:11705.0-12459.0 and MN908947.3:14331.0-14581.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:14331.0-14581.0 and MN908947.3:15617.0-16226.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:37 - TrimOlap] MN908947.3:16241.0-16708.0 and MN908947.3:17634.0-18170.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:18088.0-18691.0 and MN908947.3:19588.0-20318.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:19839.0-20587.0 and MN908947.3:21641.0-21941.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:21641.0-21941.0 and MN908947.3:22090.0-22593.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:38 - TrimOlap] MN908947.3:22090.0-22593.0 and MN908947.3:23554.0-23999.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:39 - TrimOlap] MN908947.3:23554.0-23999.0 and MN908947.3:25708.0-26428.0 cannot be concatenated as there is no overlap and they do not abut.
  [00:09:48 - ValidArgs] Reads will be filtered to only those with RG tag: 2
  [00:09:48 - Annotate] Getting chrom coordinates
  [00:09:53 - Annotate] Processing chunk with coordinates: MN908947.3:1113-26810
  Lines   total/split/realigned/skipped:        4/0/0/0
  Note: the --sample option not given, applying all records regardless of the genotype
  Applied 4 variants
  Running: minimap2 -a -x map-ont -t 2 primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95_barcode95.fastq | samtools view -bS -F 4 - | samtools sort -o barcode95.sorted.bam -
  Running: samtools index barcode95.sorted.bam
  Running: align_trim barcode95.sorted.bam primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.scheme.bed  --normalise 200 --start --report barcode95.alignreport.txt 2> barcode95.alignreport.er | samtools sort -T barcode95 - -o barcode95.trimmed.rg.sorted.bam
  Running: align_trim barcode95.sorted.bam primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.scheme.bed  --normalise 200 --report barcode95.alignreport.txt 2> barcode95.alignreport.er | samtools sort -T barcode95 - -o barcode95.primertrimmed.rg.sorted.bam
  Running: samtools index barcode95.trimmed.rg.sorted.bam
  Running: samtools index barcode95.primertrimmed.rg.sorted.bam
  Running: medaka consensus --model r941_min_hac_variant_g507 --threads 2 --chunk_len 800 --chunk_ovlp 400 --RG 1 barcode95.trimmed.rg.sorted.bam barcode95.1.hdf
  Running: medaka variant --gvcf primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95.1.hdf barcode95.1.vcf
  Running: medaka tools annotate --pad 25 --RG 1 barcode95.1.vcf primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95.trimmed.rg.sorted.bam tmp.medaka-annotate.vcf
  Running: mv tmp.medaka-annotate.vcf barcode95.1.vcf
  Running: medaka consensus --model r941_min_hac_variant_g507 --threads 2 --chunk_len 800 --chunk_ovlp 400 --RG 2 barcode95.trimmed.rg.sorted.bam barcode95.2.hdf
  Running: medaka variant --gvcf primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95.2.hdf barcode95.2.vcf
  Running: medaka tools annotate --pad 25 --RG 2 barcode95.2.vcf primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95.trimmed.rg.sorted.bam tmp.medaka-annotate.vcf
  Running: mv tmp.medaka-annotate.vcf barcode95.2.vcf
  Running: artic_vcf_merge barcode95 primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.scheme.bed 2> barcode95.primersitereport.txt 1:barcode95.1.vcf 2:barcode95.2.vcf
  Running: bgzip -f barcode95.merged.vcf
  Running: tabix -p vcf barcode95.merged.vcf.gz
  Running: bcftools filter -e "ALT='.'" barcode95.merged.vcf.gz | bgzip > tmp.vcf.gz
  Running: mv barcode95.merged.vcf.gz barcode95.merged.gvcf.vcf.gz
  Running: mv barcode95.merged.vcf.gz.tbi barcode95.merged.gvcf.vcf.gz.tbi
  Running: mv tmp.vcf.gz barcode95.merged.vcf.gz
  Running: tabix -p vcf barcode95.merged.vcf.gz
  Running: artic-tools check_vcf --summaryOut barcode95.vcfreport.txt barcode95.merged.vcf.gz primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.scheme.bed 2> barcode95.vcfcheck.log
  Running: artic_vcf_filter --medaka --min-depth 20 --min-qual 20 --nanopolish-qual-cov-ratio 3.0 barcode95.merged.vcf.gz barcode95.pass.vcf barcode95.fail.vcf
  Running: artic_make_depth_mask --depth 20 primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95.primertrimmed.rg.sorted.bam barcode95.coverage_mask.txt
  Running: artic_mask primer_schemes/SARS-CoV-2/Midnight-ONT/V3/SARS-CoV-2.reference.fasta barcode95.coverage_mask.txt barcode95.fail.vcf barcode95.preconsensus.fasta
  Running: bcftools norm --check-ref x --fasta-ref barcode95.preconsensus.fasta -O z -o barcode95.pass.vcf.gz barcode95.pass.vcf
  Running: tabix -p vcf barcode95.pass.vcf.gz
  Running: bcftools consensus -f barcode95.preconsensus.fasta barcode95.pass.vcf.gz -m barcode95.coverage_mask.txt -o barcode95.consensus.fasta
  Running: artic_fasta_header barcode95.consensus.fasta "barcode95/ARTIC/medaka"
  Running hacked up minion.py
  sed: couldn't open temporary file ./sed3unqLx: Permission denied

Work dir:
  /Users/corneliusromer/Downloads/BJGMC_ONT_Run71/work/48/570c50ab60c9945c2210fcc2d18302

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
corneliusroemer commented 1 year ago

I'm now running outside of the download folder, same issue - so it's not that I'm running in Downloads folder.

Maybe something is up with the sequence - but then there should be more meaningful errors?

When looking at the temporary file that's trying to be executed: it's empty! So no surprise permission is denied.

mattdmem commented 1 year ago

Hey @corneliusroemer - has this issue surfaced again or can I close?