Apple M1 & Docker Issue

[vestalgd]

I've run the 'nextflow run Cecret.nf -c configs/docker.config' command to test 3 samples and I have encountered an error.

N E X T F L O W ~ version 21.10.1 Launching Cecret.nf [hungry_faggin] - revision: 9cede0c1f7 Currently using the Cecret workflow for use with amplicon-based Illumina hybrid library prep on MiSeq

Author: Erin Young email: eriny@utah.gov Version: v.2.2.20211220

Fastq file found : 7093-MS-1_80 Fastq file found : 7093-MS-1_7 Fastq file found : 7093-MS-1_81 The maximum number of CPUS used in this workflow is 8 The files and directory for results is /Users/vestalg/Cecret-master 2/cecret A table summarizing results will be created: /Users/vestalg/Cecret-master 2/cecret/summary.txt and /Users/vestalg/Cecret-master 2/cecret_run_results.txt

Reference Genome : /Users/vestalg/Cecret-master 2/configs/MN908947.3.fasta GFF file for Reference Genome : /Users/vestalg/Cecret-master 2/configs/MN908947.3.gff Primer BedFile : /Users/vestalg/Cecret-master 2/configs/artic_V3_nCoV-2019.bed Amplicon BedFile : /Users/vestalg/Cecret-master 2/configs/nCoV-2019.insert.bed

executor > local (6) [be/112020] process > fastqc (7093-MS-1_81) [ 0%] 0 of 3 [09/afc9fb] process > seqyclean (7093-MS-1_81) [ 0%] 0 of 3 executor > local (6) [be/112020] process > fastqc (7093-MS-1_81) [ 0%] 0 of 3 [09/afc9fb] process > seqyclean (7093-MS-1_81) [ 25%] 1 of 4, failed: 1, r.. [- ] process > bwa - [- ] process > sort - executor > local (10) [04/185650] process > fastqc (7093-MS-1_80) [ 40%] 2 of 5, failed: 2, r.. [e8/d25ab3] process > seqyclean (7093-MS-1_80) [ 50%] 3 of 6, failed: 3, r.. [- ] process > bwa - [- ] process > sort - [- ] process > filter - [- ] process > ivar_trim - [- ] process > ivar_variants - [- ] process > ivar_consensus - [- ] process > fasta_prep - [- ] process > bcftools_variants - [- ] process > bamsnap - [- ] process > samtools_stats - executor > local (12) [35/a63224] process > fastqc (7093-MS-1_81) [ 50%] 3 of 6, failed: 3, r.. [ed/d3f88a] process > seqyclean (7093-MS-1_80) [ 50%] 3 of 6, failed: 3, r.. [- ] process > bwa - [- ] process > sort - [- ] process > filter - [- ] process > ivar_trim - [- ] process > ivar_variants - [- ] process > ivar_consensus - [- ] process > fasta_prep - [- ] process > bcftools_variants - [- ] process > bamsnap - [- ] process > samtools_stats - [- ] process > samtools_coverage - [- ] process > samtools_flagstat - executor > local (12) [35/a63224] process > fastqc (7093-MS-1_81) [ 50%] 3 of 6, failed: 3, r.. [ed/d3f88a] process > seqyclean (7093-MS-1_80) [ 50%] 3 of 6, failed: 3, r.. [- ] process > bwa - [- ] process > sort - [- ] process > filter - [- ] process > ivar_trim - [- ] process > ivar_variants - [- ] process > ivar_consensus - [- ] process > fasta_prep - [- ] process > bcftools_variants - [- ] process > bamsnap - [- ] process > samtools_stats - [- ] process > samtools_coverage - [- ] process > samtools_flagstat - [- ] process > samtools_depth - [- ] process > kraken2 - [- ] process > bedtools_multicov - [- ] process > samtools_ampliconstats - [- ] process > samtools_plot_ampliconstats - [- ] process > pangolin - [- ] process > nextclade - [- ] process > vadr - [- ] process > summary - [- ] process > combine_results - [09/afc9fb] NOTE: Process seqyclean (7093-MS-1_81) terminated with an error exit status (127) -- Execution is retried (1) [b3/7b5c47] NOTE: Process fastqc (7093-MS-1_7) terminated with an error exit status (127) -- Execution is retried (1) [ae/30a22f] NOTE: Process fastqc (7093-MS-1_80) terminated with an error exit status (127) -- Execution is retried (1) [af/4e7788] NOTE: Process seqyclean (7093-MS-1_7) terminated with an error exit status (127) -- Execution is retried (1) [e8/d25ab3] NOTE: Process seqyclean (7093-MS-1_80) terminated with an error exit status (127) -- Execution is retried (1) [be/112020] NOTE: Process fastqc (7093-MS-1_81) terminated with an error exit status (127) -- Execution is retried (1) Error executing process > 'fastqc (7093-MS-1_80)'

Caused by: Process fastqc (7093-MS-1_80) terminated with an error exit status (127)

Command executed:

mkdir -p fastqc logs/fastqc log_file=logs/fastqc/7093-MS-1_80.f574859a-d7c7-4223-a402-3fd94fdd9e50.log err_file=logs/fastqc/7093-MS-1_80.f574859a-d7c7-4223-a402-3fd94fdd9e50.err

time stamp + capturing tool versions

date | tee -a $log_file $err_file > /dev/null fastqc --version >> $log_file

fastqc --outdir fastqc --threads 1 7093-MS-1_80_S1_L005_R1_001.fastq.gz 7093-MS-1_80_S1_L005_R2_001.fastq.gz 2>> $err_file >> $log_file

zipped_fastq=($(ls fastqc/*fastqc.zip) "")

raw_1=$(unzip -p ${zipped_fastq[0]} /fastqc_data.txt | grep "Total Sequences" | awk '{ print $3 }' ) raw_2=NA if [ -f "${zipped_fastq[1]}" ] ; then raw_2=$(unzip -p fastqc/fastqc.zip */fastqc_data.txt | grep "Total Sequences" | awk '{ print $3 }' ) ; fi

if [ -z "$raw_1" ] ; then raw_1="0" ; fi if [ -z "$raw_2" ] ; then raw_2="0" ; fi

Command exit status: 127

Command output: (empty) executor > local (12) [04/185650] process > fastqc (7093-MS-1_80) [ 66%] 4 of 6, failed: 4, r.. [ed/d3f88a] process > seqyclean (7093-MS-1_80) [ 50%] 3 of 6, failed: 3, r.. [- ] process > bwa - [- ] process > sort - [- ] process > filter - [- ] process > ivar_trim - [- ] process > ivar_variants - [- ] process > ivar_consensus - [- ] process > fasta_prep - [- ] process > bcftools_variants - [- ] process > bamsnap - [- ] process > samtools_stats - [- ] process > samtools_coverage - [- ] process > samtools_flagstat - [- ] process > samtools_depth - [- ] process > kraken2 - [- ] process > bedtools_multicov - [- ] process > samtools_ampliconstats - [- ] process > samtools_plot_ampliconstats - [- ] process > pangolin - [- ] process > nextclade - [- ] process > vadr - [- ] process > summary - [- ] process > combine_results - [09/afc9fb] NOTE: Process seqyclean (7093-MS-1_81) terminated with an error exit status (127) -- Execution is retried (1) [b3/7b5c47] NOTE: Process fastqc (7093-MS-1_7) terminated with an error exit status (127) -- Execution is retried (1) [ae/30a22f] NOTE: Process fastqc (7093-MS-1_80) terminated with an error exit status (127) -- Execution is retried (1) [af/4e7788] NOTE: Process seqyclean (7093-MS-1_7) terminated with an error exit status (127) -- Execution is retried (1) [e8/d25ab3] NOTE: Process seqyclean (7093-MS-1_80) terminated with an error exit status (127) -- Execution is retried (1) [be/112020] NOTE: Process fastqc (7093-MS-1_81) terminated with an error exit status (127) -- Execution is retried (1) Error executing process > 'fastqc (7093-MS-1_80)'

Caused by: Process fastqc (7093-MS-1_80) terminated with an error exit status (127)

Command executed:

mkdir -p fastqc logs/fastqc log_file=logs/fastqc/7093-MS-1_80.f574859a-d7c7-4223-a402-3fd94fdd9e50.log err_file=logs/fastqc/7093-MS-1_80.f574859a-d7c7-4223-a402-3fd94fdd9e50.err

time stamp + capturing tool versions

date | tee -a $log_file $err_file > /dev/null fastqc --version >> $log_file

fastqc --outdir fastqc --threads 1 7093-MS-1_80_S1_L005_R1_001.fastq.gz 7093-MS-1_80_S1_L005_R2_001.fastq.gz 2>> $err_file >> $log_file

zipped_fastq=($(ls fastqc/*fastqc.zip) "")

raw_1=$(unzip -p ${zipped_fastq[0]} /fastqc_data.txt | grep "Total Sequences" | awk '{ print $3 }' ) raw_2=NA if [ -f "${zipped_fastq[1]}" ] ; then raw_2=$(unzip -p fastqc/fastqc.zip */fastqc_data.txt | grep "Total Sequences" | awk '{ print $3 }' ) ; fi

if [ -z "$raw_1" ] ; then raw_1="0" ; fi if [ -z "$raw_2" ] ; then raw_2="0" ; fi

Command exit status: 127

Command output: (empty)

Command error: WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested /bin/bash: line 0: export: `2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/FastQC/': not a valid identifier .command.sh: line 2: mkdir: command not found

Work dir: /Users/vestalg/Cecret-master 2/work/04/185650992b0ea86d57547adb49c938

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

Is this related to the Apple M1 chip I am using? I've attempted to use Option 1 and Option 2 to run from Docker and Singularity and I encounter an issue with the StaphB FastQC Docker container:

FATAL: Unable to pull docker://staphb/fastqc:latest: conveyor failed to get: no descriptor found for reference "5d67fad373325a597d0eb75d0986ad7dd78f8b2b67e78435d9176c434a8cec04"

Is this an issue with the Apple M1 chips?

[erinyoung]

That is a good question!

I'll look into this and will hopefully have a more solid answer for you on Monday.

[erinyoung]

I passed your issue on to staphb's dockerhub, since most of the containers used are from there.

https://github.com/StaPH-B/docker-builds/issues/276

[erinyoung]

I tried to replicate your error. From the other issue comments, sounded like this error might be something that is related to slurm, so I installed slurm, but was unable to replicate your error. 😢

@MillironX said the following that I'm hopeful is helpful to you

Anecdotally, I've found the issue is less prevalent on Singularity 3.8.x vs. earlier releases, but I don't have any numbers to back that, and there might be other factors involved.

Also anecdotally, I've had better success specifying the domain for all images, e.g. docker.io/staphb/fastqc:latest rather than just staphb/fastqc:latest.

In any case, this can usually be resolved by running singularity pull docker://staphb/fastqc:latest (or whatever container name is giving trouble) prior to executing the Nextflow pipeline. Singularity will then pick up on the cached image and transfer it to Nextflow.

I'm hopeful that this helps.