Suspect that that conda environment is not being created within the docker profile

DanBeaton commented 2 years ago

Hello

I am attempting to run the pipeline on my Mac using the docker profile. The process fails with the warning:

WARN: There's no process matching config selector: fastqc

The environmentl.yml file provided does not include the fastqc package.

name: nanortax channels:

conda-forge
bioconda
defaults
anaconda dependencies:
conda-forge::python
conda-forge::markdown
conda-forge::pymdown-extensions
conda-forge::pygments
anaconda::scikit-bio
bioconda::csvtk
bioconda::taxonkit
fastp
kraken2
centrifuge
pandas
blast
gawk



daniellebeaton2@danielles-iMac sediment_16S % /Users/daniellebeaton2/nextflow/nextflow /Users/daniellebeaton2/git/NanoRTax/main.nf -qs 1 -profile docker --reads '/Volumes/NanoporeM1C/Nanopore_sequencer_data/Cobalt/sediment_16S/20220613_1441_MC-112933_FAS49702_5a81a876/fastq_pass/barcode09/consolidated09.fastq' --blast_db --blast_taxdb
N E X T F L O W  ~  version 21.10.6
Launching `/Users/daniellebeaton2/git/NanoRTax/main.nf` [ridiculous_stallman] - revision: 8698db181b
WARN: Access to undefined parameter `multiqc_config` -- Initialise it to a default value eg. `params.multiqc_config = some_value`
WARN: Access to undefined parameter `reads_rt` -- Initialise it to a default value eg. `params.reads_rt = some_value`
WARN: Access to undefined parameter `kraken` -- Initialise it to a default value eg. `params.kraken = some_value`
WARN: Access to undefined parameter `centrifuge` -- Initialise it to a default value eg. `params.centrifuge = some_value`
WARN: Access to undefined parameter `blast` -- Initialise it to a default value eg. `params.blast = some_value`
----------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/rtnanopipeline v1.0dev
----------------------------------------------------

Run Name          : ridiculous_stallman
Reads             : /Volumes/NanoporeM1C/Nanopore_sequencer_data/Cobalt/sediment_16S/20220613_1441_MC-112933_FAS49702_5a81a876/fastq_pass/barcode09/consolidated09.fastq
Max Resources     : 128 GB memory, 16 cpus, 10d time per job
Container         : docker - hecrp/nanortax:latest
Output dir        : ./results
Launch dir        : /Volumes/NanoporeM1C/Nanopore_sequencer_data/Cobalt/sediment_16S
Working dir       : /Volumes/NanoporeM1C/Nanopore_sequencer_data/Cobalt/sediment_16S/work
Script dir        : /Users/daniellebeaton2/git/NanoRTax
User              : daniellebeaton2
Config Profile    : docker
----------------------------------------------------
WARN: Access to undefined parameter `hostnames` -- Initialise it to a default value eg. `params.hostnames = some_value`
executor >  local (2)
[99/029dbb] process > QC (1)               [  0%] 0 of 1
[-        ] process > qc_reporting         -
[-        ] process > read_binning_blast   -
[-        ] process > agg_blast            -
executor >  local (2)
[99/029dbb] process > QC (1)               [100%] 1 of 1, failed: 1 ✘
[-        ] process > qc_reporting         -
[-        ] process > read_binning_blast   -
[-        ] process > agg_blast            -
[-        ] process > blast_push           -
[-        ] process > agg_blast_diversity  -
[ee/b1a023] process > output_documentation [100%] 1 of 1 ✔
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/rtnanopipeline] Pipeline completed with errors-
WARN: There's no process matching config selector: fastqc
Error executing process > 'QC (1)'

Caused by:
  Process `QC (1)` terminated with an error exit status (255)

Command executed:

  barcode=$(basename $(dirname /Volumes/NanoporeM1C/Nanopore_sequencer_data/Cobalt/sediment_16S/20220613_1441_MC-112933_FAS49702_5a81a876/fastq_pass/barcode09/consolidated09.fastq))
  fastp -i /Volumes/NanoporeM1C/Nanopore_sequencer_data/Cobalt/sediment_16S/20220613_1441_MC-112933_FAS49702_5a81a876/fastq_pass/barcode09/consolidated09.fastq -q 8 -l 1400 --length_limit 1700 -o $barcode\_qced_reads.fastq --json $barcode\_qc_report.txt
  head -n30 $barcode\_qc_report.txt | sed '30s/,/\n}/' > $barcode\_qc_report.json
  echo "}" >> $barcode\_qc_report.json

Command exit status:
  255

Command output:
  (empty)

Command error:
  ERROR: Failed to open file: /Volumes/NanoporeM1C/Nanopore_sequencer_data/Cobalt/sediment_16S/20220613_1441_MC-112933_FAS49702_5a81a876/fastq_pass/barcode09/consolidated09.fastq

Work dir:
  /Volumes/NanoporeM1C/Nanopore_sequencer_data/Cobalt/sediment_16S/work/99/029dbba425a071ed77419df3da574f

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

DanBeaton commented 2 years ago

Additional information: after running NanoRTax using the conda profile I get the following error

Caused by: java.nio.channels.OverlappingFileLockException

Creating the conda environment manually still produces the error. Running the fastp code on its own does work.

I suspect that the conda environment is not being created under either the docker or the conda profile.

vicho33 commented 2 years ago

I have a similar problem:

Launching `main.nf` [soggy_gilbert] DSL1 - revision: 3ef505f614 WARN: Access to undefined parameter `multiqc_config` -- Initialise it to a default value eg. `params.multiqc_config = some_value` WARN: Access to undefined parameter `reads_rt` -- Initialise it to a default value eg. `params.reads_rt = some_value` WARN: Access to undefined parameter `kraken` -- Initialise it to a default value eg. `params.kraken = some_value` WARN: Access to undefined parameter `centrifuge` -- Initialise it to a default value eg. `params.centrifuge = some_value` WARN: Access to undefined parameter `blast` -- Initialise it to a default value eg. `params.blast = some_value`

                                    ,--./,-.
    ___     __   __   __   ___     /,-._.--~'

|\ | | / `/ \ |) | } { | \| | \, \/ | \ |___ \`-.,-`-,` .,._,' nf-core/rtnanopipeline v1.0dev

Run Name : soggy_gilbert Reads : /Users/vicentearriagada/Documents/prueba_segunda_bar/Segunda_seq_16s/fastq_pass/barcode01/FAT29459_pass_barcode01_0ae29989_0.fastq Max Resources : 128 GB memory, 16 cpus, 10d time per job Output dir : /Users/vicentearriagada/centrigufe/results_seg_bar Launch dir : /Users/vicentearriagada/centrigufe/NanoRTax Working dir : /Users/vicentearriagada/centrigufe/NanoRTax/work Script dir : /Users/vicentearriagada/centrigufe/NanoRTax User : vicentearriagada Config Profile : conda

WARN: Access to undefined parameter hostnames -- Initialise it to a default value eg. params.hostnames = some_value executor > local (2) executor > local (2) [5f/3b9906] process > QC (1) [100%] 1 of 1, failed: 1 ✘ [- ] process > qc_reporting - [- ] process > read_binning_kraken - [- ] process > agg_kraken - [- ] process > kraken_push - [- ] process > agg_kraken_diversity - [57/a71840] process > output_documentation [100%] 1 of 1, failed: 1 ✘ Execution cancelled -- Finishing pending tasks before exit -[nf-core/rtnanopipeline] Pipeline completed with errors- WARN: There's no process matching config selector: get_software_versions Error executing process > 'QC (1)'

Caused by: Process QC (1) terminated with an error exit status (127)

Command executed:

barcode=$(basename $(dirname /Users/vicentearriagada/Documents/prueba_segunda_bar/Segunda_seq_16s/fastq_pass/barcode01/FAT29459_pass_barcode01_0ae29989_0.fastq)) fastp -i /Users/vicentearriagada/Documents/prueba_segunda_bar/Segunda_seq_16s/fastq_pass/barcode01/FAT29459_pass_barcode01_0ae29989_0.fastq -q 8 -l 1400 --length_limit 1700 -o $barcode_qced_reads.fastq --json $barcode_qc_report.txt head -n30 $barcode_qc_report.txt | sed '30s/,/\n}/' > $barcode_qc_report.json echo "}" >> $barcode_qc_report.json

Command exit status: 127

Command output: (empty)

Command error: .command.sh: line 3: fastp: command not found

Work dir: /Users/vicentearriagada/centrigufe/NanoRTax/work/5f/3b99062b898c54cdc215c80464c3fb

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run I don't know that to do

genomicsITER / NanoRTax