search

bactopia / bactopia

A flexible pipeline for complete analysis of bacterial genomes
https://bactopia.github.io
MIT License
398 stars 69 forks source link

[question] Bactopia Assembly Failing #350

Closed erin-thei closed 5 months ago

erin-thei commented 2 years ago

Hi Rob,

Screen Shot 2022-07-27 at 5 07 37 PM

I am using Bactopia v2.0.3, running the command "bactopia --fastqs fastqs.txt --datasets datasets/ --species "Klebsiella pneumoniae" --coverage 100 --nfconfig /mnt/isilon/bactopia_respublica.config --cleanup_workdir --outdir klebsiella_outdir --resume". I am working on the login node of the cluster and trying to run approx. 360 samples.

We had an outage on our cluster yesterday that caused Bactopia to fail in the middle of the run. It got through the gather samples and qc reads steps already, and was working through assembling the genomes as well as some of the other steps. Using the -resume flag I re-ran the same command yesterday. However, upon looking at the progress, it looks like a lot of the assemblies are failing. Additionally, the total number of assemblies is more than the total number of samples I have. Have you seen this before?

In addition, the warning "Bactopia: Assemble Genome terminated with error exit status 225 -- execution retried (1)" has come up multiple times. Do you know what this is related to and how this affects the run? I wasn't able to find anything in the Issues section.

I've had multiple successful (although smaller) runs using this version of Bactopia, so everything was working well.

Lastly, would it be better to start this run from scratch due to this behavior that was likely caused by the outage? Or what would you suggest?

Thanks!

rpetit3 commented 2 years ago

Hmmmm, Did this fail almost immeadiately after rerunning with -resume? Or are the running then failing overtime?

I think it would be useful to dig into the work directory so see if maybe we can find something. Let me know if you need my help on this!

erin-thei commented 2 years ago

After rerunning with -resume, is when it seems that all of the genome assemblies are failing. Prior to the outage (and thus re-running resume), the genomes were seemed to be assembling successfully. However, when I looked again earlier today, it said that 119 out of 119 genomes, failed which didn't seem right.

Just trying to learn more as well, what does that error code mean?

Also, is there something specific that I should look for in the work directory? Not sure where to start to look.

Thanks for your help!

rpetit3 commented 2 years ago

In the image about try to find work/7d/b13cbd...

In that folder will be a .command.err, .command.out and. shovill.log file I wonder if we might find something useful in those

erin-thei commented 2 years ago

Ah, I see. Thanks for pointing that out! Still very new to all of this so learning as I go.

Screen Shot 2022-07-27 at 7 02 42 PM

After looking at those files (although truthfully not exactly knowing what I am looking for), something that stood out to me was in the .command.err file. It says Error: NCBI C++ Exception. Not sure what that totally means, but that seemed to be the only Error message in any of the three files. The rest of them had finished successfully.

rpetit3 commented 2 years ago

you did great! Keep going and you'll see "You do not have write permissions to blastdb"

You might have found a bug! Remind how are you running this (Docker, Singularity or Conda)? As well as the type of system (was it a slurm cluster?)

rpetit3 commented 2 years ago

For some reason it looks likt its failing here: https://github.com/bactopia/bactopia/blob/master/modules/local/bactopia/assemble_genome/main.nf#L138-L139

erin-thei commented 2 years ago

Hmm, very interesting! I'm running this on Conda and the system is a slurm cluster.

I also know I mentioned this before, but I'm using v2.0.3. It's worked for me on multiple runs prior to this set of samples so I'm wondering what's changed that's causing this. I know just because things worked before doesn't mean they'll never fail, but just very interesting especially to someone just first starting out haha.

erin-thei commented 2 years ago

So upon further inspection it looks like there is a difference in naming of the assembly that might be causing the issues downstream. Looking at the shovill log, the assembly was completed successfully and was labeled as contigs.fa, but there was no contigs.fa in that folder. Looking at that same folder /work/7d/b13cbd.../results, the assembly is there but labelled under the prefix (I.350.2.Colony10.fna). I'm wondering if this may be causing some type of issue... What do you think?

Screen Shot 2022-07-28 at 9 46 52 AM Screen Shot 2022-07-28 at 9 46 24 AM
rpetit3 commented 2 years ago

That should actually be ok since I rename the contigs.fa to <SAMPLE_NAME>.fna

I wonder what happens if you try rerunning just one of these samples without resuming.

erin-thei commented 2 years ago

Okay, I will give that a try. In the meantime, I do have another question...

I am trying to learn how to run this on the cluster and not on my head node. Like I mentioned earlier, the system is a slurm cluster. I tried to run this command: bactopia --fastqs fastqs.txt --datasets datasets/ --species 'Klebsiella pneumoniae' --coverage 100 --nfconfig /mnt/isilon/marc_genomics/bactopia_respublica.config -profile respublica --cleanup_workdir --outdir bactopia_outdir. However, it failed with the following error message. Do you know how I could resolve this?

(bactopia) [theillere@reslnvvhpc0153 klebsiella-bactopia]$ bactopia --fastqs fastqs.txt --datasets datasets/ --species 'Klebsiella pneumoniae' --coverage 100 --nfconfig /mnt/isilon/marc_genomics/bactopia_respublica.config -profile respublica --cleanup_workdir --outdir bactopia_outdir N E X T F L O W ~ version 22.04.0 Launching /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/main.nf [ecstatic_solvay] DSL2 - revision: bd75c553f9

| | _ | | __ (_)
| ' \ / ` |/ | / | ' | |/ ` |
| |) | (| | (| || () | |) | | (| |
|./ _,|_|__/| ./||_,| |_|
bactopia v2.0.3 Bactopia is a flexible pipeline for complete analysis of bacterial genomes.

Core Nextflow options runName : ecstatic_solvay launchDir : /scr1/users/theillere/klebsiella-bactopia workDir : /scr1/users/theillere/klebsiella-bactopia/work projectDir : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x userName : theillere profile : respublica configFiles : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/nextflow.config

Required Parameters fastqs : fastqs.txt

Dataset Parameters datasets : datasets/ species : Klebsiella pneumoniae

Optional Parameters outdir : bactopia_outdir

Nextflow Configuration Parameters nfconfig : /mnt/isilon/marc_genomics/bactopia_respublica.config cleanup_workdir : true

Nextflow Profile Parameters condadir : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/conda/envs registry : quay singularity_cache: /home/theillere/.bactopia/singularity

!! Only displaying parameters that differ from the pipeline defaults !!

If you use bactopia for your analysis please cite:

If something looks wrong, now's your chance to back out (CTRL+C 3 times). Sleeping for 5 seconds...

executor > slurm (100) [7a/f9e916] process > BACTOPIA:GATHER_SAMPLES (F22.f.DNA.1.5) [ 0%] 0 of 360 [- ] process > BACTOPIA:QC_READS - [- ] process > BACTOPIA:ASSEMBLE_GENOME - [- ] process > BACTOPIA:ASSEMBLY_QC - [- ] process > BACTOPIA:ANNOTATE_GENOME - [- ] process > BACTOPIA:MINMER_SKETCH - [- ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE - [- ] process > BACTOPIA:ARIBA_ANALYSIS - [- ] process > BACTOPIA:MINMER_QUERY - [- ] process > BACTOPIA:BLAST - [- ] process > BACTOPIA:CALL_VARIANTS - [- ] process > BACTOPIA:MAPPING_QUERY - [- ] process > BACTOPIA:SEQUENCE_TYPE - [- ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS - Error executing process > 'BACTOPIA:GATHER_SAMPLES (F.1.Colony1)'

Caused by: Process BACTOPIA:GATHER_SAMPLES (F.1.Colony1) terminated with an error exit status (1)

Command executed:

MERGED="multiple-read-sets-merged.txt" mkdir -p fastqs mkdir -p extra

if [ "paired-end" == "paired-end" ]; then

Paired-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1_R1.fastq.gz
  cp -L 001-r2 fastqs/F.1.Colony1_R2.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "single-end" ]; then

Single-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "ont" ]; then

Nanopore reads

  cp -L 001-r1 fastqs/F.1.Colony1.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "hybrid" ]; then

Paired-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1_R1.fastq.gz
  cp -L 001-r2 fastqs/F.1.Colony1_R2.fastq.gz
  cp -L EMPTY_EXTRA extra/F.1.Colony1.fastq.gz

elif [ "paired-end" == "merge-pe" ] || [ "paired-end" == "hybrid-merge-pe" ]; then

Merge Paired-End Reads

  echo "This sample had reads merged." > ${MERGED}
  echo "R1:" >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1_R1.fastq.gz
  echo "Merged R1:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1_R1.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

  echo "R2:" >> ${MERGED}
  find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1_R2.fastq.gz
  echo "Merged R2:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1_R2.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

  if [ "paired-end" == "hybrid-merge-pe" ]; then
      cp -L EMPTY_EXTRA extra/F.1.Colony1.fastq.gz
  else
      touch extra/empty.fna.gz
  fi

elif [ "paired-end" == "merge-se" ]; then

Merge Single-End Reads

  echo "This sample had reads merged." > ${MERGED}
  echo "SE:" >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1.fastq.gz
  echo "Merged SE:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1.fastq.gz | awk '{print $5"   "$9}' >> ${MERGED}

  touch extra/empty.fna.gz

elif [ "paired-end" == "sra_accession" ]; then if [ "1" == "3" ]; then echo "Unable to download F.1.Colony1 from both SRA and ENA 3 times. This may or may not be a temporary connection issue. Rather than stop the whole Bactopia run, further analysis of F.1.Colony1 will be discontinued." | sed 's/^\s*//' > F.1.Colony1-fastq-download-error.txt exit else

Download accession from ENA/SRA

      fastq-dl F.1.Colony1 SRA                 --cpus 8                 --outdir fastqs/                 --group_by_experiment                 --is_experiment                 --ftp_only
      touch extra/empty.fna.gz
  fi

elif [ "false" == "true" ]; then if [ "paired-end" == "assembly_accession" ]; then if [ "1" == "3" ]; then touch extra/empty.fna.gz echo "Unable to download F.1.Colony1 from NCBI Assembly 3 times. This may or may not be a temporary connection issue. Rather than stop the whole Bactopia run, further analysis of F.1.Colony1 will be discontinued." | sed 's/^\s*//' > F.1.Colony1-assembly-download-error.txt exit else

Verify Assembly accession

          check-assembly-accession.py F.1.Colony1 > accession.txt 2> check-assembly-accession.txt

          if [ -s "accession.txt" ]; then
              # Download from NCBI assembly and simulate reads
              mkdir fasta/
              ncbi-genome-download bacteria -o ./ -F fasta -p 8                                                 -s null -A accession.txt -r 50 
              find . -name "*F.1.Colony1*.fna.gz" | xargs -I {} mv {} fasta/
              rename 's/(GC[AF]_\d+).*/$1.fna.gz/' fasta/*
              gzip -cd fasta/F.1.Colony1.fna.gz > F.1.Colony1-art.fna
              rm check-assembly-accession.txt
          else
              mv check-assembly-accession.txt F.1.Colony1-assembly-accession-error.txt
              exit
          fi
      fi
  elif [ "paired-end" == "assembly" ]; then
      if [ "false" == "true" ]; then
          gzip -cd EMPTY_EXTRA > F.1.Colony1-art.fna
      else 
          cat EMPTY_EXTRA > F.1.Colony1-art.fna
      fi
  fi

  # Simulate reads from assembly, reads are 250bp without errors
  art_illumina -p -ss MSv3 -l 250 -m 400 -s 30 --fcov 150 -ir 0 -ir2 0 -dr 0 -dr2 0 -rs 42                         -na -qL 33 -qU 40 -o F.1.Colony1_R --id F.1.Colony1 -i F.1.Colony1-art.fna

  mv F.1.Colony1_R1.fq fastqs/F.1.Colony1_R1.fastq
  mv F.1.Colony1_R2.fq fastqs/F.1.Colony1_R2.fastq
  pigz -p 8 --fast fastqs/*.fastq
  cp F.1.Colony1-art.fna extra/F.1.Colony1.fna
  pigz -p 8 --best extra/F.1.Colony1.fna

fi

Validate input FASTQs

if [ "false" == "false" ]; then ERROR=0

Check paired-end reads have same read counts

  OPTS="--sample F.1.Colony1 --min_basepairs 2241820 --min_reads 7472 --min_proportion 0.5"
  if [ -f  "fastqs/F.1.Colony1_R2.fastq.gz" ]; then
      # Paired-end
      gzip -cd fastqs/F.1.Colony1_R1.fastq.gz | fastq-scan > r1.json
      gzip -cd fastqs/F.1.Colony1_R2.fastq.gz | fastq-scan > r2.json
      if ! reformat.sh in1=fastqs/F.1.Colony1_R1.fastq.gz in2=fastqs/F.1.Colony1_R2.fastq.gz qin=auto out=/dev/null 2> F.1.Colony1-paired-end-error.txt; then
          ERROR=1
          echo "F.1.Colony1 FASTQs contains an error. Please check the input FASTQs.
              Further analysis is discontinued." |                 sed 's/^\s*//' >> F.1.Colony1-paired-end-error.txt
      else
          rm -f F.1.Colony1-paired-end-error.txt
      fi

      if ! check-fastqs.py --fq1 r1.json --fq2 r2.json ${OPTS}; then
          ERROR=1
      fi
      rm r1.json r2.json
  else
      # Single-end
      gzip -cd fastqs/F.1.Colony1.fastq.gz | fastq-scan > r1.json
      if ! check-fastqs.py --fq1 r1.json ${OPTS}; then
          ERROR=1
      fi
      rm r1.json
  fi

  # Failed validations so, let's keep them from continuing
  if [ "${ERROR}" -eq "1" ]; then
      mv fastqs/ failed-tests-fastqs/
  fi

fi

Estimate Genome Size

GENOME_SIZE_OUTPUT="F.1.Colony1-genome-size.txt" if [ "0" == "0" ]; then if [ "false" == "true" ]; then

Use the total assembly size as the genome size

      stats.sh in=extra/F.1.Colony1.fna.gz | grep All | awk '{print $5}' | sed 's/,//g' > ${GENOME_SIZE_OUTPUT}
  else
      FASTQS=""
      if [ -f  "fastqs/F.1.Colony1_R2.fastq.gz" ]; then
          FASTQS="-r fastqs/F.1.Colony1_R1.fastq.gz fastqs/F.1.Colony1_R2.fastq.gz"
      else
          FASTQS="fastqs/F.1.Colony1.fastq.gz"
      fi

      # First Pass
      mash sketch -o test -k 31 -m 3 ${FASTQS} 2>&1 |                 grep "Estimated genome size:" |                 awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
      rm -rf test.msh
      ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`

      # Check if second pass is needed
      if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ] || [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
          # Probably high coverage, try increasing number of kmer copies to 10
          M="-m 10"
          if [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
              # Probably low coverage, try decreasing the number of kmer copies to 1
              M="-m 1"
          fi
          mash sketch -o test -k 31 ${M} ${FASTQS} 2>&1 |                     grep "Estimated genome size:" |                     awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
          rm -rf test.msh
      fi
  fi

  # Check final estimate
  ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`
  if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ]; then
      rm ${GENOME_SIZE_OUTPUT}
      echo "F.1.Colony1 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) exceeds the maximum
              allowed genome size (18040666 bp). If this is unexpected, please
              investigate F.1.Colony1 to determine a cause (e.g. metagenomic, contaminants, etc...).
              Otherwise, adjust the --max_genome_size parameter to fit your need. Further analysis
              of F.1.Colony1 will be discontinued." |             sed 's/^\s*//' > F.1.Colony1-genome-size-error.txt
  elif [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
      rm ${GENOME_SIZE_OUTPUT}
      echo "F.1.Colony1 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) is less than the minimum
              allowed genome size (100000 bp). If this is unexpected, please
              investigate F.1.Colony1 to determine a cause (e.g. metagenomic, contaminants, etc...).
              Otherwise, adjust the --min_genome_size parameter to fit your need. Further analysis
              of F.1.Colony1 will be discontinued." |             sed 's/^\s*//' > F.1.Colony1-genome-size-error.txt
  fi

else

Use the genome size given by the user. (Should be >= 0)

  echo "0" > ${GENOME_SIZE_OUTPUT}

fi

Capture versions

cat <<-END_VERSIONS > versions.yml "BACTOPIA:GATHER_SAMPLES": art: $(echo $(art_illumina --help 2>&1) | sed 's/^.Version //;s/ .$//') fastq-dl: $(echo $(fastq-dl --version 2>&1) | sed 's/fastq-dl //') fastq-scan: $(echo $(fastq-scan -v 2>&1) | sed 's/fastq-scan //') mash: $(echo $(mash --version 2>&1)) ncbi-genome-download: $(echo $(ncbi-genome-download --version 2>&1)) pigz: $(echo $(pigz --version 2>&1) | sed 's/pigz //') END_VERSIONS

Command exit status: 1

Command output: (empty)

Command error: cp: cannot create regular file 'fastqs/F.1.Colony1_R1.fastq.gz/001-r1': No such file or directory

Work dir: /scr1/users/theillere/klebsiella-bactopia/work/24/89689b614fa2a1a006f3744190d487

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

Bactopia Execution Summary
---------------------------
Bactopia Version : 2.0.3
Nextflow Version : 22.04.0
Command Line     : nextflow run /home/theillere/miniconda3/envs/bactopia//share/bactopia-2.0.x/main.nf -w /scr1/users/theillere/klebsiella-bactopia/work/ --fastqs fastqs.txt --datasets datasets/ --species 'Klebsiella pneumoniae' --coverage 100 --nfconfig /mnt/isilon/marc_genomics/bactopia_respublica.config -profile respublica --cleanup_workdir --outdir bactopia_outdir
Resumed          : false
Completed At     : 2022-07-29T09:16:56.338070-04:00
Duration         : 24.9s
Success          : false
Exit Code        : 1
Error Report     : Error executing process > 'BACTOPIA:GATHER_SAMPLES (F.1.Colony1)'

Caused by: Process BACTOPIA:GATHER_SAMPLES (F.1.Colony1) terminated with an error exit status (1)

Command executed:

MERGED="multiple-read-sets-merged.txt" mkdir -p fastqs mkdir -p extra

if [ "paired-end" == "paired-end" ]; then

Paired-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1_R1.fastq.gz
  cp -L 001-r2 fastqs/F.1.Colony1_R2.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "single-end" ]; then

Single-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "ont" ]; then

Nanopore reads

  cp -L 001-r1 fastqs/F.1.Colony1.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "hybrid" ]; then

Paired-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1_R1.fastq.gz
  cp -L 001-r2 fastqs/F.1.Colony1_R2.fastq.gz
  cp -L EMPTY_EXTRA extra/F.1.Colony1.fastq.gz

elif [ "paired-end" == "merge-pe" ] || [ "paired-end" == "hybrid-merge-pe" ]; then

Merge Paired-End Reads

  echo "This sample had reads merged." > ${MERGED}
  echo "R1:" >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1_R1.fastq.gz
  echo "Merged R1:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1_R1.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

  echo "R2:" >> ${MERGED}
  find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1_R2.fastq.gz
  echo "Merged R2:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1_R2.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

  if [ "paired-end" == "hybrid-merge-pe" ]; then
      cp -L EMPTY_EXTRA extra/F.1.Colony1.fastq.gz
  else
      touch extra/empty.fna.gz
  fi

elif [ "paired-end" == "merge-se" ]; then

Merge Single-End Reads

  echo "This sample had reads merged." > ${MERGED}
  echo "SE:" >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1.fastq.gz
  echo "Merged SE:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1.fastq.gz | awk '{print $5"   "$9}' >> ${MERGED}

  touch extra/empty.fna.gz

elif [ "paired-end" == "sra_accession" ]; then if [ "1" == "3" ]; then echo "Unable to download F.1.Colony1 from both SRA and ENA 3 times. This may or may not be a temporary connection issue. Rather than stop the whole Bactopia run, further analysis of F.1.Colony1 will be discontinued." | sed 's/^\s*//' > F.1.Colony1-fastq-download-error.txt exit else

Download accession from ENA/SRA

      fastq-dl F.1.Colony1 SRA                 --cpus 8                 --outdir fastqs/                 --group_by_experiment                 --is_experiment                 --ftp_only
      touch extra/empty.fna.gz
  fi

elif [ "false" == "true" ]; then if [ "paired-end" == "assembly_accession" ]; then if [ "1" == "3" ]; then touch extra/empty.fna.gz echo "Unable to download F.1.Colony1 from NCBI Assembly 3 times. This may or may not be a temporary connection issue. Rather than stop the whole Bactopia run, further analysis of F.1.Colony1 will be discontinued." | sed 's/^\s*//' > F.1.Colony1-assembly-download-error.txt exit else

Verify Assembly accession

          check-assembly-accession.py F.1.Colony1 > accession.txt 2> check-assembly-accession.txt

          if [ -s "accession.txt" ]; then
              # Download from NCBI assembly and simulate reads
              mkdir fasta/
              ncbi-genome-download bacteria -o ./ -F fasta -p 8                                                 -s null -A accession.txt -r 50 
              find . -name "*F.1.Colony1*.fna.gz" | xargs -I {} mv {} fasta/
              rename 's/(GC[AF]_\d+).*/$1.fna.gz/' fasta/*
              gzip -cd fasta/F.1.Colony1.fna.gz > F.1.Colony1-art.fna
              rm check-assembly-accession.txt
          else
              mv check-assembly-accession.txt F.1.Colony1-assembly-accession-error.txt
              exit
          fi
      fi
  elif [ "paired-end" == "assembly" ]; then
      if [ "false" == "true" ]; then
          gzip -cd EMPTY_EXTRA > F.1.Colony1-art.fna
      else 
          cat EMPTY_EXTRA > F.1.Colony1-art.fna
      fi
  fi

  # Simulate reads from assembly, reads are 250bp without errors
  art_illumina -p -ss MSv3 -l 250 -m 400 -s 30 --fcov 150 -ir 0 -ir2 0 -dr 0 -dr2 0 -rs 42                         -na -qL 33 -qU 40 -o F.1.Colony1_R --id F.1.Colony1 -i F.1.Colony1-art.fna

  mv F.1.Colony1_R1.fq fastqs/F.1.Colony1_R1.fastq
  mv F.1.Colony1_R2.fq fastqs/F.1.Colony1_R2.fastq
  pigz -p 8 --fast fastqs/*.fastq
  cp F.1.Colony1-art.fna extra/F.1.Colony1.fna
  pigz -p 8 --best extra/F.1.Colony1.fna

fi

Validate input FASTQs

if [ "false" == "false" ]; then ERROR=0

Check paired-end reads have same read counts

  OPTS="--sample F.1.Colony1 --min_basepairs 2241820 --min_reads 7472 --min_proportion 0.5"
  if [ -f  "fastqs/F.1.Colony1_R2.fastq.gz" ]; then
      # Paired-end
      gzip -cd fastqs/F.1.Colony1_R1.fastq.gz | fastq-scan > r1.json
      gzip -cd fastqs/F.1.Colony1_R2.fastq.gz | fastq-scan > r2.json
      if ! reformat.sh in1=fastqs/F.1.Colony1_R1.fastq.gz in2=fastqs/F.1.Colony1_R2.fastq.gz qin=auto out=/dev/null 2> F.1.Colony1-paired-end-error.txt; then
          ERROR=1
          echo "F.1.Colony1 FASTQs contains an error. Please check the input FASTQs.
              Further analysis is discontinued." |                 sed 's/^\s*//' >> F.1.Colony1-paired-end-error.txt
      else
          rm -f F.1.Colony1-paired-end-error.txt
      fi

      if ! check-fastqs.py --fq1 r1.json --fq2 r2.json ${OPTS}; then
          ERROR=1
      fi
      rm r1.json r2.json
  else
      # Single-end
      gzip -cd fastqs/F.1.Colony1.fastq.gz | fastq-scan > r1.json
      if ! check-fastqs.py --fq1 r1.json ${OPTS}; then
          ERROR=1
      fi
      rm r1.json
  fi

  # Failed validations so, let's keep them from continuing
  if [ "${ERROR}" -eq "1" ]; then
      mv fastqs/ failed-tests-fastqs/
  fi

fi

Estimate Genome Size

GENOME_SIZE_OUTPUT="F.1.Colony1-genome-size.txt" if [ "0" == "0" ]; then if [ "false" == "true" ]; then

Use the total assembly size as the genome size

      stats.sh in=extra/F.1.Colony1.fna.gz | grep All | awk '{print $5}' | sed 's/,//g' > ${GENOME_SIZE_OUTPUT}
  else
      FASTQS=""
      if [ -f  "fastqs/F.1.Colony1_R2.fastq.gz" ]; then
          FASTQS="-r fastqs/F.1.Colony1_R1.fastq.gz fastqs/F.1.Colony1_R2.fastq.gz"
      else
          FASTQS="fastqs/F.1.Colony1.fastq.gz"
      fi

      # First Pass
      mash sketch -o test -k 31 -m 3 ${FASTQS} 2>&1 |                 grep "Estimated genome size:" |                 awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
      rm -rf test.msh
      ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`

      # Check if second pass is needed
      if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ] || [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
          # Probably high coverage, try increasing number of kmer copies to 10
          M="-m 10"
          if [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
              # Probably low coverage, try decreasing the number of kmer copies to 1
              M="-m 1"
          fi
          mash sketch -o test -k 31 ${M} ${FASTQS} 2>&1 |                     grep "Estimated genome size:" |                     awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
          rm -rf test.msh
      fi
  fi

  # Check final estimate
  ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`
  if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ]; then
      rm ${GENOME_SIZE_OUTPUT}
      echo "F.1.Colony1 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) exceeds the maximum
              allowed genome size (18040666 bp). If this is unexpected, please
              investigate F.1.Colony1 to determine a cause (e.g. metagenomic, contaminants, etc...).
              Otherwise, adjust the --max_genome_size parameter to fit your need. Further analysis
              of F.1.Colony1 will be discontinued." |             sed 's/^\s*//' > F.1.Colony1-genome-size-error.txt
  elif [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
      rm ${GENOME_SIZE_OUTPUT}
      echo "F.1.Colony1 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) is less than the minimum
              allowed genome size (100000 bp). If this is unexpected, please
              investigate F.1.Colony1 to determine a cause (e.g. metagenomic, contaminants, etc...).
              Otherwise, adjust the --min_genome_size parameter to fit your need. Further analysis
              of F.1.Colony1 will be discontinued." |             sed 's/^\s*//' > F.1.Colony1-genome-size-error.txt
  fi

else

Use the genome size given by the user. (Should be >= 0)

  echo "0" > ${GENOME_SIZE_OUTPUT}

fi executor > slurm (101) [03/aa3afd] process > BACTOPIA:GATHER_SAMPLES (F.12.Colony1) [ 5%] 14 of 260, failed: 14 [- ] process > BACTOPIA:QC_READS - [- ] process > BACTOPIA:ASSEMBLE_GENOME - [- ] process > BACTOPIA:ASSEMBLY_QC - [- ] process > BACTOPIA:ANNOTATE_GENOME - [- ] process > BACTOPIA:MINMER_SKETCH - [- ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE - [- ] process > BACTOPIA:ARIBA_ANALYSIS - [- ] process > BACTOPIA:MINMER_QUERY - [- ] process > BACTOPIA:BLAST - [- ] process > BACTOPIA:CALL_VARIANTS - [- ] process > BACTOPIA:MAPPING_QUERY - [- ] process > BACTOPIA:SEQUENCE_TYPE - [- ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS - Error executing process > 'BACTOPIA:GATHER_SAMPLES (F.1.Colony1)'

Caused by: Process BACTOPIA:GATHER_SAMPLES (F.1.Colony1) terminated with an error exit status (1)

Command executed:

MERGED="multiple-read-sets-merged.txt" mkdir -p fastqs mkdir -p extra

if [ "paired-end" == "paired-end" ]; then

Paired-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1_R1.fastq.gz
  cp -L 001-r2 fastqs/F.1.Colony1_R2.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "single-end" ]; then

Single-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "ont" ]; then

Nanopore reads

  cp -L 001-r1 fastqs/F.1.Colony1.fastq.gz
  touch extra/empty.fna.gz

elif [ "paired-end" == "hybrid" ]; then

Paired-End Reads

  cp -L 001-r1 fastqs/F.1.Colony1_R1.fastq.gz
  cp -L 001-r2 fastqs/F.1.Colony1_R2.fastq.gz
  cp -L EMPTY_EXTRA extra/F.1.Colony1.fastq.gz

elif [ "paired-end" == "merge-pe" ] || [ "paired-end" == "hybrid-merge-pe" ]; then

Merge Paired-End Reads

  echo "This sample had reads merged." > ${MERGED}
  echo "R1:" >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1_R1.fastq.gz
  echo "Merged R1:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1_R1.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

  echo "R2:" >> ${MERGED}
  find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1_R2.fastq.gz
  echo "Merged R2:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1_R2.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

  if [ "paired-end" == "hybrid-merge-pe" ]; then
      cp -L EMPTY_EXTRA extra/F.1.Colony1.fastq.gz
  else
      touch extra/empty.fna.gz
  fi

elif [ "paired-end" == "merge-se" ]; then

Merge Single-End Reads

  echo "This sample had reads merged." > ${MERGED}
  echo "SE:" >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
  find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony1.fastq.gz
  echo "Merged SE:" >> ${MERGED}
  ls -l fastqs/F.1.Colony1.fastq.gz | awk '{print $5"   "$9}' >> ${MERGED}

  touch extra/empty.fna.gz

elif [ "paired-end" == "sra_accession" ]; then if [ "1" == "3" ]; then echo "Unable to download F.1.Colony1 from both SRA and ENA 3 times. This may or may not be a temporary connection issue. Rather than stop the whole Bactopia run, further analysis of F.1.Colony1 will be discontinued." | sed 's/^\s*//' > F.1.Colony1-fastq-download-error.txt exit else

Download accession from ENA/SRA

      fastq-dl F.1.Colony1 SRA                 --cpus 8                 --outdir fastqs/                 --group_by_experiment                 --is_experiment                 --ftp_only
      touch extra/empty.fna.gz
  fi

elif [ "false" == "true" ]; then if [ "paired-end" == "assembly_accession" ]; then if [ "1" == "3" ]; then touch extra/empty.fna.gz echo "Unable to download F.1.Colony1 from NCBI Assembly 3 times. This may or may not be a temporary connection issue. Rather than stop the whole Bactopia run, further analysis of F.1.Colony1 will be discontinued." | sed 's/^\s*//' > F.1.Colony1-assembly-download-error.txt exit else

Verify Assembly accession

          check-assembly-accession.py F.1.Colony1 > accession.txt 2> check-assembly-accession.txt

          if [ -s "accession.txt" ]; then
              # Download from NCBI assembly and simulate reads
              mkdir fasta/
              ncbi-genome-download bacteria -o ./ -F fasta -p 8                                                 -s null -A accession.txt -r 50 
              find . -name "*F.1.Colony1*.fna.gz" | xargs -I {} mv {} fasta/
              rename 's/(GC[AF]_\d+).*/$1.fna.gz/' fasta/*
              gzip -cd fasta/F.1.Colony1.fna.gz > F.1.Colony1-art.fna
              rm check-assembly-accession.txt
          else
              mv check-assembly-accession.txt F.1.Colony1-assembly-accession-error.txt
              exit
          fi
      fi
  elif [ "paired-end" == "assembly" ]; then
      if [ "false" == "true" ]; then
          gzip -cd EMPTY_EXTRA > F.1.Colony1-art.fna
      else 
          cat EMPTY_EXTRA > F.1.Colony1-art.fna
      fi
  fi

  # Simulate reads from assembly, reads are 250bp without errors
  art_illumina -p -ss MSv3 -l 250 -m 400 -s 30 --fcov 150 -ir 0 -ir2 0 -dr 0 -dr2 0 -rs 42                         -na -qL 33 -qU 40 -o F.1.Colony1_R --id F.1.Colony1 -i F.1.Colony1-art.fna

  mv F.1.Colony1_R1.fq fastqs/F.1.Colony1_R1.fastq
  mv F.1.Colony1_R2.fq fastqs/F.1.Colony1_R2.fastq
  pigz -p 8 --fast fastqs/*.fastq
  cp F.1.Colony1-art.fna extra/F.1.Colony1.fna
  pigz -p 8 --best extra/F.1.Colony1.fna

fi

Validate input FASTQs

if [ "false" == "false" ]; then ERROR=0

Check paired-end reads have same read counts

  OPTS="--sample F.1.Colony1 --min_basepairs 2241820 --min_reads 7472 --min_proportion 0.5"
  if [ -f  "fastqs/F.1.Colony1_R2.fastq.gz" ]; then
      # Paired-end
      gzip -cd fastqs/F.1.Colony1_R1.fastq.gz | fastq-scan > r1.json
      gzip -cd fastqs/F.1.Colony1_R2.fastq.gz | fastq-scan > r2.json
      if ! reformat.sh in1=fastqs/F.1.Colony1_R1.fastq.gz in2=fastqs/F.1.Colony1_R2.fastq.gz qin=auto out=/dev/null 2> F.1.Colony1-paired-end-error.txt; then
          ERROR=1
          echo "F.1.Colony1 FASTQs contains an error. Please check the input FASTQs.
              Further analysis is discontinued." |                 sed 's/^\s*//' >> F.1.Colony1-paired-end-error.txt
      else
          rm -f F.1.Colony1-paired-end-error.txt
      fi

      if ! check-fastqs.py --fq1 r1.json --fq2 r2.json ${OPTS}; then
          ERROR=1
      fi
      rm r1.json r2.json
  else
      # Single-end
      gzip -cd fastqs/F.1.Colony1.fastq.gz | fastq-scan > r1.json
      if ! check-fastqs.py --fq1 r1.json ${OPTS}; then
          ERROR=1
      fi
      rm r1.json
  fi

  # Failed validations so, let's keep them from continuing
  if [ "${ERROR}" -eq "1" ]; then
      mv fastqs/ failed-tests-fastqs/
  fi

fi

Estimate Genome Size

GENOME_SIZE_OUTPUT="F.1.Colony1-genome-size.txt" if [ "0" == "0" ]; then if [ "false" == "true" ]; then

Use the total assembly size as the genome size

      stats.sh in=extra/F.1.Colony1.fna.gz | grep All | awk '{print $5}' | sed 's/,//g' > ${GENOME_SIZE_OUTPUT}
  else
      FASTQS=""
      if [ -f  "fastqs/F.1.Colony1_R2.fastq.gz" ]; then
          FASTQS="-r fastqs/F.1.Colony1_R1.fastq.gz fastqs/F.1.Colony1_R2.fastq.gz"
      else
          FASTQS="fastqs/F.1.Colony1.fastq.gz"
      fi

      # First Pass
      mash sketch -o test -k 31 -m 3 ${FASTQS} 2>&1 |                 grep "Estimated genome size:" |                 awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
      rm -rf test.msh
      ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`

      # Check if second pass is needed
      if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ] || [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
          # Probably high coverage, try increasing number of kmer copies to 10
          M="-m 10"
          if [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
              # Probably low coverage, try decreasing the number of kmer copies to 1
              M="-m 1"
          fi
          mash sketch -o test -k 31 ${M} ${FASTQS} 2>&1 |                     grep "Estimated genome size:" |                     awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
          rm -rf test.msh
      fi
  fi

  # Check final estimate
  ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`
  if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ]; then
      rm ${GENOME_SIZE_OUTPUT}
      echo "F.1.Colony1 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) exceeds the maximum
              allowed genome size (18040666 bp). If this is unexpected, please
              investigate F.1.Colony1 to determine a cause (e.g. metagenomic, contaminants, etc...).
              Otherwise, adjust the --max_genome_size parameter to fit your need. Further analysis
              of F.1.Colony1 will be discontinued." |             sed 's/^\s*//' > F.1.Colony1-genome-size-error.txt
  elif [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
      rm ${GENOME_SIZE_OUTPUT}
      echo "F.1.Colony1 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) is less than the minimum
              allowed genome size (100000 bp). If this is unexpected, please
              investigate F.1.Colony1 to determine a cause (e.g. metagenomic, contaminants, etc...).
              Otherwise, adjust the --min_genome_size parameter to fit your need. Further analysis
              of F.1.Colony1 will be discontinued." |             sed 's/^\s*//' > F.1.Colony1-genome-size-error.txt
  fi

else

Use the genome size given by the user. (Should be >= 0)

  echo "0" > ${GENOME_SIZE_OUTPUT}

fi

Capture versions

cat <<-END_VERSIONS > versions.yml "BACTOPIA:GATHER_SAMPLES": art: $(echo $(art_illumina --help 2>&1) | sed 's/^.Version //;s/ .$//') fastq-dl: $(echo $(fastq-dl --version 2>&1) | sed 's/fastq-dl //') fastq-scan: $(echo $(fastq-scan -v 2>&1) | sed 's/fastq-scan //') mash: $(echo $(mash --version 2>&1)) ncbi-genome-download: $(echo $(ncbi-genome-download --version 2>&1)) pigz: $(echo $(pigz --version 2>&1) | sed 's/pigz //') END_VERSIONS

Command exit status: 1

Command output: (empty)

Command error: cp: cannot create regular file 'fastqs/F.1.Colony1_R1.fastq.gz/001-r1': No such file or directory

Work dir: /scr1/users/theillere/klebsiella-bactopia/work/24/89689b614fa2a1a006f3744190d487

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

(bactopia) [theillere@reslnvvhpc0153 klebsiella-bactopia]$

erin-thei commented 2 years ago
Screen Shot 2022-07-29 at 3 15 24 PM

This is what my config file looks like

rpetit3 commented 2 years ago

Good news it ran on slurm! I think you just need to tell it to use singularity

Here's an example config I'm using on our cluster: https://github.com/bactopia/bactopia/blob/master/conf/profiles/arcc.config

erin-thei commented 2 years ago

Might sound like silly question but I would tell it to use singularity even if i'm running this on conda?

rpetit3 commented 2 years ago

When you log in, use conda as you normally do. But we'll tell Nextflow to use singularity when it submits jobs to your cluster.

Let's try -profile singularity,respublica if that works we can update your respublica profile

erin-thei commented 2 years ago

I tried the same command as above but instead used -profile singularity, respublica but it failed. Here's the full output:

(bactopia) [theillere@reslnvvhpc0153 klebsiella-bactopia]$ bactopia --fastqs fastqs.txt --datasets datasets/ --species 'Klebsiella pneumoniae' --coverage 100 --nfconfig /mnt/isilon/marc_genomics/bactopia_respublica.config -profile singularity,respublica --cleanup_workdir --outdir bactopia_outdir
N E X T F L O W  ~  version 22.04.0
Launching `/home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/main.nf` [evil_rubens] DSL2 - revision: bd75c553f9

---------------------------------------------
   _                _              _             
  | |__   __ _  ___| |_ ___  _ __ (_) __ _       
  | '_ \ / _` |/ __| __/ _ \| '_ \| |/ _` |   
  | |_) | (_| | (__| || (_) | |_) | | (_| |      
  |_.__/ \__,_|\___|\__\___/| .__/|_|\__,_| 
                            |_|                  
  bactopia v2.0.3
  Bactopia is a flexible pipeline for complete analysis of bacterial genomes. 
---------------------------------------------
Core Nextflow options
  runName          : evil_rubens
  containerEngine  : singularity
  container        : quay.io/bactopia/bactopia:2.0.3
  launchDir        : /scr1/users/theillere/klebsiella-bactopia
  workDir          : /scr1/users/theillere/klebsiella-bactopia/work
  projectDir       : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x
  userName         : theillere
  profile          : singularity,respublica
  configFiles      : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/nextflow.config

Required Parameters
  fastqs           : fastqs.txt

Dataset Parameters
  datasets         : datasets/
  species          : Klebsiella pneumoniae

Optional Parameters
  outdir           : bactopia_outdir

Nextflow Configuration Parameters
  nfconfig         : /mnt/isilon/marc_genomics/bactopia_respublica.config
  cleanup_workdir  : true

Nextflow Profile Parameters
  condadir         : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/conda/envs
  registry         : quay
  singularity_cache: /home/theillere/.bactopia/singularity

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------
If you use bactopia for your analysis please cite:

* Bactopia
  https://doi.org/10.1128/mSystems.00190-20

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://bactopia.github.io/acknowledgements/
--------------------------------------------------------------------
Found 1 Antimicrobial resistance datasets
    /scr1/users/theillere/klebsiella-bactopia/datasets/antimicrobial-resistance/amrfinderdb.tar.gz
Found 2 ARIBA datasets
    /scr1/users/theillere/klebsiella-bactopia/datasets/ariba/card.tar.gz
    /scr1/users/theillere/klebsiella-bactopia/datasets/ariba/vfdb_core.tar.gz
Found 4 minmer sketches/signatures
    /scr1/users/theillere/klebsiella-bactopia/datasets/minmer/mash-refseq-k21.msh
    /scr1/users/theillere/klebsiella-bactopia/datasets/minmer/sourmash-genbank-k21.json.gz
    /scr1/users/theillere/klebsiella-bactopia/datasets/minmer/sourmash-genbank-k31.json.gz
    /scr1/users/theillere/klebsiella-bactopia/datasets/minmer/sourmash-genbank-k51.json.gz
Found Prokka proteins file
    /scr1/users/theillere/klebsiella-bactopia/datasets/species-specific/klebsiella-pneumoniae/annotation/klebsiella-pneumoniae.faa
Found Mash Sketch of auto variant calling
    datasets//species-specific/klebsiella-pneumoniae/minmer/refseq-genomes.msh
Found 1 MLST datasets
    /scr1/users/theillere/klebsiella-bactopia/datasets/species-specific/klebsiella-pneumoniae/mlst/default.tar.gz
Found 1 reference genomes
    /scr1/users/theillere/klebsiella-bactopia/datasets/species-specific/klebsiella-pneumoniae/minmer/refseq-genomes.msh
Found 0 bp for genome size, it will be estimated.

If something looks wrong, now's your chance to back out (CTRL+C 3 times). 
Sleeping for 5 seconds...
--------------------------------------------------------------------
executor >  local (2)
executor >  local (2)
executor >  local (2)
[18/d6c833] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony1) [  2%] 2 of 93, failed: 2, retries: 2
executor >  local (3)
[e6/9d97b2] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony3) [  1%] 2 of 117, failed: 2, retries: 2
executor >  local (3)
[e6/9d97b2] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony3) [  2%] 3 of 132, failed: 3, retries: 3
[-        ] process > BACTOPIA:QC_READS                     -
executor >  local (4)
[fa/81a4d9] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony4) [  2%] 3 of 146, failed: 3, retries: 3
[-        ] process > BACTOPIA:QC_READS                     -
executor >  local (4)
[fa/81a4d9] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony4) [  1%] 3 of 161, failed: 3, retries: 3
[-        ] process > BACTOPIA:QC_READS                     -
executor >  local (5)
[a1/110c50] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony5) [  2%] 4 of 179, failed: 4, retries: 4
[-        ] process > BACTOPIA:QC_READS                     -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME              -
executor >  local (5)
[a1/110c50] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony5) [  1%] 4 of 204, failed: 4, retries: 4
[-        ] process > BACTOPIA:QC_READS                     -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME              -
executor >  local (6)
[52/28108b] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony6) [  2%] 5 of 225, failed: 5, retries: 5
[-        ] process > BACTOPIA:QC_READS                     -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME              -
[-        ] process > BACTOPIA:ASSEMBLY_QC                  -
executor >  local (6)
[52/28108b] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony6) [  2%] 5 of 247, failed: 5, retries: 5
[-        ] process > BACTOPIA:QC_READS                     -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME              -
[-        ] process > BACTOPIA:ASSEMBLY_QC                  -
executor >  local (7)
[2c/81f9a4] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony7) [  2%] 6 of 269, failed: 6, retries: 6
[-        ] process > BACTOPIA:QC_READS                     -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME              -
[-        ] process > BACTOPIA:ASSEMBLY_QC                  -
[-        ] process > BACTOPIA:ANNOTATE_GENOME              -
executor >  local (7)
[2c/81f9a4] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony7) [  2%] 6 of 293, failed: 6, retries: 6
[-        ] process > BACTOPIA:QC_READS                     -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME              -
[-        ] process > BACTOPIA:ASSEMBLY_QC                  -
[-        ] process > BACTOPIA:ANNOTATE_GENOME              -
executor >  local (8)
[dd/6690ad] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony8) [  2%] 7 of 320, failed: 7, retries: 7
[-        ] process > BACTOPIA:QC_READS                     -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME              -
[-        ] process > BACTOPIA:ASSEMBLY_QC                  -
[-        ] process > BACTOPIA:ANNOTATE_GENOME              -
[-        ] process > BACTOPIA:MINMER_SKETCH                -
executor >  local (8)
[dd/6690ad] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony8) [  2%] 7 of 346, failed: 7, retries: 7
[-        ] process > BACTOPIA:QC_READS                     -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME              -
[-        ] process > BACTOPIA:ASSEMBLY_QC                  -
[-        ] process > BACTOPIA:ANNOTATE_GENOME              -
[-        ] process > BACTOPIA:MINMER_SKETCH                -
executor >  local (9)
[61/5a840d] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony2) [  2%] 8 of 368, failed: 8, retries: 8
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
executor >  local (10)
[ed/a3045d] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony1) [  2%] 9 of 369, failed: 9, retries: 9
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
executor >  local (11)
[b1/907ab5] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony3) [  2%] 10 of 370, failed: 10, retries: 10
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
executor >  local (12)
[2c/67a688] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony6) [  2%] 11 of 371, failed: 11, retries: 11
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
executor >  local (13)
[d4/05ac13] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony4) [  3%] 12 of 372, failed: 12, retries: 12
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
executor >  local (14)
[9c/882aed] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony5) [  3%] 13 of 373, failed: 13, retries: 13
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
executor >  local (14)
[9c/882aed] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony5) [  3%] 13 of 373, failed: 13, retries: 13
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
executor >  local (15)
[64/0f45f3] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony7) [  3%] 14 of 374, failed: 14, retries: 14
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
executor >  local (16)
[47/b55766] process > BACTOPIA:GATHER_SAMPLES (F.10.Colony8) [  4%] 15 of 375, failed: 15, retries: 15
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
executor >  local (17)
[ab/6e71a3] process > BACTOPIA:GATHER_SAMPLES (F.11.Colony1) [  4%] 16 of 376, failed: 16, retries: 16
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Executioexecutor >  local (18)
[6a/6c3a49] process > BACTOPIA:GATHER_SAMPLES (F.11.Colony3) [  4%] 17 of 377, failed: 17, retries: 17
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Executioexecutor >  local (19)
[cd/33e26f] process > BACTOPIA:GATHER_SAMPLES (F.11.Colony2) [  4%] 18 of 378, failed: 18, retries: 18
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Executioexecutor >  local (20)
[6d/53e37a] process > BACTOPIA:GATHER_SAMPLES (F.11.Colony4) [  5%] 19 of 379, failed: 19, retries: 19
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Executioexecutor >  local (21)
[5e/291202] process > BACTOPIA:GATHER_SAMPLES (F.11.Colony5) [  5%] 20 of 380, failed: 20, retries: 20
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Executioexecutor >  local (22)
[c4/7984bc] process > BACTOPIA:GATHER_SAMPLES (F.11.Colony6) [  5%] 21 of 381, failed: 21, retries: 21
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Executioexecutor >  local (23)
[46/5298ca] process > BACTOPIA:GATHER_SAMPLES (F.11.Colony8) [  5%] 22 of 382, failed: 22, retries: 22
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Executioexecutor >  local (25)
[fc/5d5eaa] process > BACTOPIA:GATHER_SAMPLES (F.12.Colony1) [  6%] 24 of 384, failed: 24, retries: 24
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Executiexecutor >  local (26)
[ef/281a9e] process > BACTOPIA:GATHER_SAMPLES (F.12.Colony2) [  6%] 25 of 385, failed: 25, retries: 25
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Executiexecutor >  local (27)
[7d/ab2b18] process > BACTOPIA:GATHER_SAMPLES (F.12.Colony3) [  6%] 26 of 386, failed: 26, retries: 26
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Executiexecutor >  local (28)
[4a/befbf7] process > BACTOPIA:GATHER_SAMPLES (F.12.Colony5) [  6%] 27 of 387, failed: 27, retries: 27
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Executiexecutor >  local (29)
[f7/561d56] process > BACTOPIA:GATHER_SAMPLES (F.12.Colony4) [  7%] 28 of 388, failed: 28, retries: 28
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Executiexecutor >  local (31)
[84/c01490] process > BACTOPIA:GATHER_SAMPLES (F.12.Colony7) [  7%] 30 of 390, failed: 30, retries: 30
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Executiexecutor >  local (32)
[a9/b3919d] process > BACTOPIA:GATHER_SAMPLES (F.12.Colony8) [  7%] 31 of 391, failed: 31, retries: 31
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Executiexecutor >  local (33)
[78/11f751] process > BACTOPIA:GATHER_SAMPLES (F.3.Colony1)  [  8%] 32 of 392, failed: 32, retries: 32
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Executiexecutor >  local (34)
[f4/587355] process > BACTOPIA:GATHER_SAMPLES (F.3.Colony2)  [  8%] 33 of 393, failed: 33, retries: 33
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Executiexecutor >  local (35)
[d5/cf170a] process > BACTOPIA:GATHER_SAMPLES (F.3.Colony3)  [  8%] 34 of 394, failed: 34, retries: 34
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Executiexecutor >  local (36)
[1e/b507bc] process > BACTOPIA:GATHER_SAMPLES (F.3.Colony4)  [  8%] 35 of 395, failed: 35, retries: 35
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Executiexecutor >  local (37)
[89/9e885d] process > BACTOPIA:GATHER_SAMPLES (F.3.Colony5)  [  9%] 37 of 396, failed: 37, retries: 37
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Executiexecutor >  local (39)
[28/c55ac9] process > BACTOPIA:GATHER_SAMPLES (F.3.Colony6)  [  9%] 38 of 398, failed: 38, retries: 38
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Executiexecutor >  local (40)
[d7/0d284a] process > BACTOPIA:GATHER_SAMPLES (F.4.Colony1)  [  9%] 39 of 399, failed: 39, retries: 39
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Executiexecutor >  local (41)
[2f/d1a6c4] process > BACTOPIA:GATHER_SAMPLES (F.3.Colony8)  [ 10%] 40 of 400, failed: 40, retries: 40
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Executiexecutor >  local (42)
[f8/a288c4] process > BACTOPIA:GATHER_SAMPLES (F.4.Colony2)  [ 10%] 41 of 401, failed: 41, retries: 41
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Executiexecutor >  local (43)
[45/39f83a] process > BACTOPIA:GATHER_SAMPLES (F.4.Colony4)  [ 10%] 43 of 403, failed: 43, retries: 43
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Executiexecutor >  local (45)
[3f/a0c7dd] process > BACTOPIA:GATHER_SAMPLES (F.4.Colony5)  [ 10%] 44 of 404, failed: 44, retries: 44
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Executiexecutor >  local (46)
[4c/b2f2f8] process > BACTOPIA:GATHER_SAMPLES (F.4.Colony6)  [ 11%] 45 of 405, failed: 45, retries: 45
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Executiexecutor >  local (47)
[2c/8de607] process > BACTOPIA:GATHER_SAMPLES (F.5.Colony1)  [ 11%] 46 of 406, failed: 46, retries: 46
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Executiexecutor >  local (49)
[40/bd49e4] process > BACTOPIA:GATHER_SAMPLES (F.4.Colony8)  [ 11%] 48 of 408, failed: 48, retries: 48
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Executioexecutor >  local (50)
[a3/4816ba] process > BACTOPIA:GATHER_SAMPLES (F.5.Colony2)  [ 11%] 49 of 409, failed: 49, retries: 49
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Executioexecutor >  local (50)
[a3/4816ba] process > BACTOPIA:GATHER_SAMPLES (F.5.Colony2)  [ 12%] 50 of 409, failed: 50, retries: 50
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Executioexecutor >  local (51)
[ca/16d950] process > BACTOPIA:GATHER_SAMPLES (F.5.Colony3)  [ 12%] 50 of 410, failed: 50, retries: 50
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Executioexecutor >  local (52)
[14/4ade94] process > BACTOPIA:GATHER_SAMPLES (F.5.Colony4)  [ 12%] 51 of 411, failed: 51, retries: 51
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Executioexecutor >  local (53)
[fa/d8309f] process > BACTOPIA:GATHER_SAMPLES (F.5.Colony5)  [ 12%] 53 of 413, failed: 53, retries: 53
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Executioexecutor >  local (55)
[41/b66597] process > BACTOPIA:GATHER_SAMPLES (F.5.Colony7)  [ 13%] 54 of 414, failed: 54, retries: 54
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Executioexecutor >  local (56)
[fa/705527] process > BACTOPIA:GATHER_SAMPLES (F.5.Colony8)  [ 13%] 55 of 415, failed: 55, retries: 55
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d7/0d284a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony1)` terminated with an error exit status (127) -- Executioexecutor >  local (57)
[6e/024701] process > BACTOPIA:GATHER_SAMPLES (F.6.Colony1)  [ 13%] 56 of 416, failed: 56, retries: 56
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d7/0d284a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[2f/d1a6c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony8)` terminated with an error exit status (127) -- Executioexecutor >  local (58)
[87/dccc87] process > BACTOPIA:GATHER_SAMPLES (F.6.Colony2)  [ 13%] 57 of 417, failed: 57, retries: 57
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d7/0d284a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[2f/d1a6c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[f8/a288c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony2)` terminated with an error exit status (127) -- Executioexecutor >  local (59)
[b0/ae042c] process > BACTOPIA:GATHER_SAMPLES (F.6.Colony3)  [ 13%] 58 of 418, failed: 58, retries: 58
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d7/0d284a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[2f/d1a6c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[f8/a288c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[45/39f83a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony4)` terminated with an error exit status (127) -- Executioexecutor >  local (61)
[e9/101dd7] process > BACTOPIA:GATHER_SAMPLES (F.6.Colony5)  [ 14%] 60 of 420, failed: 60, retries: 60
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d7/0d284a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[2f/d1a6c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[f8/a288c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[45/39f83a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[fe/4ce8f2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[3f/a0c7dd] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony5)` terminated with an error exit status (127) -- Executioexecutor >  local (62)
[f0/f71411] process > BACTOPIA:GATHER_SAMPLES (F.1.Colony2)  [ 14%] 61 of 421, failed: 61, retries: 61
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d7/0d284a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[2f/d1a6c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[f8/a288c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[45/39f83a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[fe/4ce8f2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[3f/a0c7dd] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[4c/b2f2f8] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony6)` terminated with an error exit status (127) -- Executioexecutor >  local (63)
[fd/aca744] process > BACTOPIA:GATHER_SAMPLES (F.6.Colony7)  [ 14%] 62 of 422, failed: 62, retries: 62
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d7/0d284a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[2f/d1a6c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[f8/a288c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[45/39f83a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[fe/4ce8f2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[3f/a0c7dd] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[4c/b2f2f8] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/8de607] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[56/977539] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[40/bd49e4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[a3/4816ba] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ca/16d950] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[14/4ade94] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/d8309f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[12/672031] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[41/b66597] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/705527] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6e/024701] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[87/dccc87] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[b0/ae042c] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[7c/7c1ea0] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[e9/101dd7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f0/f71411] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (2)
Error executing process > 'BACTOPIA:GATHER_SAMPLES (F.1.Colony2)'

Caused by:
  Process requirement exceed available memory -- req: 6 GB; avail: 5.6 GB

Command executed:

  MERGED="multiple-read-sets-merged.txt"
  mkdir -p fastqs
  mkdir -p extra

  if [ "paired-end" == "paired-end" ]; then
      # Paired-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2_R1.fastq.gz
      cp -L 001-r2 fastqs/F.1.Colony2_R2.fastq.gz
      touch extra/empty.fna.gz
  elif [ "paired-end" == "single-end" ]; then
      # Single-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2.fastq.gz
      touch extra/empty.fna.gz
  elif [ "paired-end" == "ont" ]; then
      # Nanopore reads
      cp -L 001-r1 fastqs/F.1.Colony2.fastq.gz
      touch extra/empty.fna.gz
  elif  [ "paired-end" == "hybrid" ]; then
      # Paired-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2_R1.fastq.gz
      cp -L 001-r2 fastqs/F.1.Colony2_R2.fastq.gz
      cp -L EMPTY_EXTRA extra/F.1.Colony2.fastq.gz
  elif [ "paired-end" == "merge-pe" ] || [ "paired-end" == "hybrid-merge-pe" ]; then 
      # Merge Paired-End Reads
      echo "This sample had reads merged." > ${MERGED}
      echo "R1:" >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2_R1.fastq.gz
      echo "Merged R1:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2_R1.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

      echo "R2:" >> ${MERGED}
      find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2_R2.fastq.gz
      echo "Merged R2:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2_R2.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

      if [ "paired-end" == "hybrid-merge-pe" ]; then
          cp -L EMPTY_EXTRA extra/F.1.Colony2.fastq.gz
      else
          touch extra/empty.fna.gz
      fi
  elif [ "paired-end" == "merge-se" ]; then 
      # Merge Single-End Reads
      echo "This sample had reads merged." > ${MERGED}
      echo "SE:" >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2.fastq.gz
      echo "Merged SE:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2.fastq.gz | awk '{print $5"   "$9}' >> ${MERGED}

      touch extra/empty.fna.gz
  elif [ "paired-end" == "sra_accession" ]; then
      if [ "3" == "3" ]; then
          echo "Unable to download F.1.Colony2 from both SRA and ENA 3 times. This may or may 
              not be a temporary connection issue. Rather than stop the whole Bactopia run, 
              further analysis of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-fastq-download-error.txt
          exit
      else
          # Download accession from ENA/SRA
          fastq-dl F.1.Colony2 SRA                 --cpus 4                 --outdir fastqs/                 --group_by_experiment                 --is_experiment                 --ftp_only
          touch extra/empty.fna.gz
      fi 
  elif [ "false" == "true" ]; then
      if [ "paired-end" == "assembly_accession" ]; then
          if [ "3" == "3" ]; then
              touch extra/empty.fna.gz
              echo "Unable to download F.1.Colony2 from NCBI Assembly 3 times. This may or may
                  not be a temporary connection issue. Rather than stop the whole Bactopia run, 
                  further analysis of F.1.Colony2 will be discontinued." |                 sed 's/^\s*//' > F.1.Colony2-assembly-download-error.txt
              exit
          else
              # Verify Assembly accession
              check-assembly-accession.py F.1.Colony2 > accession.txt 2> check-assembly-accession.txt

              if [ -s "accession.txt" ]; then
                  # Download from NCBI assembly and simulate reads
                  mkdir fasta/
                  ncbi-genome-download bacteria -o ./ -F fasta -p 4                                                 -s null -A accession.txt -r 50 
                  find . -name "*F.1.Colony2*.fna.gz" | xargs -I {} mv {} fasta/
                  rename 's/(GC[AF]_\d+).*/$1.fna.gz/' fasta/*
                  gzip -cd fasta/F.1.Colony2.fna.gz > F.1.Colony2-art.fna
                  rm check-assembly-accession.txt
              else
                  mv check-assembly-accession.txt F.1.Colony2-assembly-accession-error.txt
                  exit
              fi
          fi
      elif [ "paired-end" == "assembly" ]; then
          if [ "false" == "true" ]; then
              gzip -cd EMPTY_EXTRA > F.1.Colony2-art.fna
          else 
              cat EMPTY_EXTRA > F.1.Colony2-art.fna
          fi
      fi

      # Simulate reads from assembly, reads are 250bp without errors
      art_illumina -p -ss MSv3 -l 250 -m 400 -s 30 --fcov 150 -ir 0 -ir2 0 -dr 0 -dr2 0 -rs 42                         -na -qL 33 -qU 40 -o F.1.Colony2_R --id F.1.Colony2 -i F.1.Colony2-art.fna

      mv F.1.Colony2_R1.fq fastqs/F.1.Colony2_R1.fastq
      mv F.1.Colony2_R2.fq fastqs/F.1.Colony2_R2.fastq
      pigz -p 4 --fast fastqs/*.fastq
      cp F.1.Colony2-art.fna extra/F.1.Colony2.fna
      pigz -p 4 --best extra/F.1.Colony2.fna
  fi

  # Validate input FASTQs
  if [ "false" == "false" ]; then
      ERROR=0
      # Check paired-end reads have same read counts
      OPTS="--sample F.1.Colony2 --min_basepairs 2241820 --min_reads 7472 --min_proportion 0.5"
      if [ -f  "fastqs/F.1.Colony2_R2.fastq.gz" ]; then
          # Paired-end
          gzip -cd fastqs/F.1.Colony2_R1.fastq.gz | fastq-scan > r1.json
          gzip -cd fastqs/F.1.Colony2_R2.fastq.gz | fastq-scan > r2.json
          if ! reformat.sh in1=fastqs/F.1.Colony2_R1.fastq.gz in2=fastqs/F.1.Colony2_R2.fastq.gz qin=auto out=/dev/null 2> F.1.Colony2-paired-end-error.txt; then
              ERROR=1
              echo "F.1.Colony2 FASTQs contains an error. Please check the input FASTQs.
                  Further analysis is discontinued." |                 sed 's/^\s*//' >> F.1.Colony2-paired-end-error.txt
          else
              rm -f F.1.Colony2-paired-end-error.txt
          fi

          if ! check-fastqs.py --fq1 r1.json --fq2 r2.json ${OPTS}; then
              ERROR=1
          fi
          rm r1.json r2.json
      else
          # Single-end
          gzip -cd fastqs/F.1.Colony2.fastq.gz | fastq-scan > r1.json
          if ! check-fastqs.py --fq1 r1.json ${OPTS}; then
              ERROR=1
          fi
          rm r1.json
      fi

      # Failed validations so, let's keep them from continuing
      if [ "${ERROR}" -eq "1" ]; then
          mv fastqs/ failed-tests-fastqs/
      fi
  fi

  # Estimate Genome Size
  GENOME_SIZE_OUTPUT="F.1.Colony2-genome-size.txt"
  if [ "0" == "0" ]; then
      if [ "false" == "true" ]; then
          # Use the total assembly size as the genome size
          stats.sh in=extra/F.1.Colony2.fna.gz | grep All | awk '{print $5}' | sed 's/,//g' > ${GENOME_SIZE_OUTPUT}
      else
          FASTQS=""
          if [ -f  "fastqs/F.1.Colony2_R2.fastq.gz" ]; then
              FASTQS="-r fastqs/F.1.Colony2_R1.fastq.gz fastqs/F.1.Colony2_R2.fastq.gz"
          else
              FASTQS="fastqs/F.1.Colony2.fastq.gz"
          fi

          # First Pass
          mash sketch -o test -k 31 -m 3 ${FASTQS} 2>&1 |                 grep "Estimated genome size:" |                 awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
          rm -rf test.msh
          ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`

          # Check if second pass is needed
          if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ] || [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
              # Probably high coverage, try increasing number of kmer copies to 10
              M="-m 10"
              if [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
                  # Probably low coverage, try decreasing the number of kmer copies to 1
                  M="-m 1"
              fi
              mash sketch -o test -k 31 ${M} ${FASTQS} 2>&1 |                     grep "Estimated genome size:" |                     awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
              rm -rf test.msh
          fi
      fi

      # Check final estimate
      ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`
      if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ]; then
          rm ${GENOME_SIZE_OUTPUT}
          echo "F.1.Colony2 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) exceeds the maximum
                  allowed genome size (18040666 bp). If this is unexpected, please
                  investigate F.1.Colony2 to determine a cause (e.g. metagenomic, contaminants, etc...).
                  Otherwise, adjust the --max_genome_size parameter to fit your need. Further analysis
                  of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-genome-size-error.txt
      elif [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
          rm ${GENOME_SIZE_OUTPUT}
          echo "F.1.Colony2 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) is less than the minimum
                  allowed genome size (100000 bp). If this is unexpected, please
                  investigate F.1.Colony2 to determine a cause (e.g. metagenomic, contaminants, etc...).
                  Otherwise, adjust the --min_genome_size parameter to fit your need. Further analysis
                  of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-genome-size-error.txt
      fi
  else
      # Use the genome size given by the user. (Should be >= 0)
      echo "0" > ${GENOME_SIZE_OUTPUT}
  fi

  # Capture versions
  cat <<-END_VERSIONS > versions.yml
  "BACTOPIA:GATHER_SAMPLES":
      art: $(echo $(art_illumina --help 2>&1) | sed 's/^.*Version //;s/ .*$//')
      fastq-dl: $(echo $(fastq-dl --version 2>&1) | sed 's/fastq-dl //')
      fastq-scan: $(echo $(fastq-scan -v 2>&1) | sed 's/fastq-scan //')
      mash: $(echo $(mash --version 2>&1))
      ncbi-genome-download: $(echo $(ncbi-genome-download --version 2>&1))
      pigz: $(echo $(pigz --version 2>&1) | sed 's/pigz //')
  END_VERSIONS

Command exit status:
  -

Command output:
  (empty)

Command error:
  env: ‘singularity’: No such file or directory

Work dir:
  /scr1/users/theillere/klebsiella-bactopia/work/cb/5dd50e99359f9b9354d2c725952f7f

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

    Bactopia Execution Summary
    ---------------------------
    Bactopia Version : 2.0.3
    Nextflow Version : 22.04.0
    Command Line     : nextflow run /home/theillere/miniconda3/envs/bactopia//share/bactopia-2.0.x/main.nf -w /scr1/users/theillere/klebsiella-bactopia/work/ --fastqs fastqs.txt --datasets datasets/ --species 'Klebsiella pneumoniae' --coverage 100 --nfconfig /mnt/isilon/marc_genomics/bactopia_respublica.config -profile singularity,respublica --cleanup_workdir --outdir bactopia_outdir
    Resumed          : false
    Completed At     : 2022-07-29T17:04:29.071850-04:00
    Duration         : 30.9s
    Success          : false
    Exit Code        : null
    Error Report     : Error executing process > 'BACTOPIA:GATHER_SAMPLES (F.1.Colony2)'

Caused by:
  Process requirement exceed available memory -- req: 6 GB; avail: 5.6 GB

Command executed:

  MERGED="multiple-read-sets-merged.txt"
  mkdir -p fastqs
  mkdir -p extra

  if [ "paired-end" == "paired-end" ]; then
      # Paired-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2_R1.fastq.gz
      cp -L 001-r2 fastqs/F.1.Colony2_R2.fastq.gz
      touch extra/empty.fna.gz
  elif [ "paired-end" == "single-end" ]; then
      # Single-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2.fastq.gz
      touch extra/empty.fna.gz
  elif [ "paired-end" == "ont" ]; then
      # Nanopore reads
      cp -L 001-r1 fastqs/F.1.Colony2.fastq.gz
      touch extra/empty.fna.gz
  elif  [ "paired-end" == "hybrid" ]; then
      # Paired-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2_R1.fastq.gz
      cp -L 001-r2 fastqs/F.1.Colony2_R2.fastq.gz
      cp -L EMPTY_EXTRA extra/F.1.Colony2.fastq.gz
  elif [ "paired-end" == "merge-pe" ] || [ "paired-end" == "hybrid-merge-pe" ]; then 
      # Merge Paired-End Reads
      echo "This sample had reads merged." > ${MERGED}
      echo "R1:" >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2_R1.fastq.gz
      echo "Merged R1:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2_R1.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

      echo "R2:" >> ${MERGED}
      find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2_R2.fastq.gz
      echo "Merged R2:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2_R2.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

      if [ "paired-end" == "hybrid-merge-pe" ]; then
          cp -L EMPTY_EXTRA extra/F.1.Colony2.fastq.gz
      else
          touch extra/empty.fna.gz
      fi
  elif [ "paired-end" == "merge-se" ]; then 
      # Merge Single-End Reads
      echo "This sample had reads merged." > ${MERGED}
      echo "SE:" >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2.fastq.gz
      echo "Merged SE:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2.fastq.gz | awk '{print $5"   "$9}' >> ${MERGED}

      touch extra/empty.fna.gz
  elif [ "paired-end" == "sra_accession" ]; then
      if [ "3" == "3" ]; then
          echo "Unable to download F.1.Colony2 from both SRA and ENA 3 times. This may or may 
              not be a temporary connection issue. Rather than stop the whole Bactopia run, 
              further analysis of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-fastq-download-error.txt
          exit
      else
          # Download accession from ENA/SRA
          fastq-dl F.1.Colony2 SRA                 --cpus 4                 --outdir fastqs/                 --group_by_experiment                 --is_experiment                 --ftp_only
          touch extra/empty.fna.gz
      fi 
  elif [ "false" == "true" ]; then
      if [ "paired-end" == "assembly_accession" ]; then
          if [ "3" == "3" ]; then
              touch extra/empty.fna.gz
              echo "Unable to download F.1.Colony2 from NCBI Assembly 3 times. This may or may
                  not be a temporary connection issue. Rather than stop the whole Bactopia run, 
                  further analysis of F.1.Colony2 will be discontinued." |                 sed 's/^\s*//' > F.1.Colony2-assembly-download-error.txt
              exit
          else
              # Verify Assembly accession
              check-assembly-accession.py F.1.Colony2 > accession.txt 2> check-assembly-accession.txt

              if [ -s "accession.txt" ]; then
                  # Download from NCBI assembly and simulate reads
                  mkdir fasta/
                  ncbi-genome-download bacteria -o ./ -F fasta -p 4                                                 -s null -A accession.txt -r 50 
                  find . -name "*F.1.Colony2*.fna.gz" | xargs -I {} mv {} fasta/
                  rename 's/(GC[AF]_\d+).*/$1.fna.gz/' fasta/*
                  gzip -cd fasta/F.1.Colony2.fna.gz > F.1.Colony2-art.fna
                  rm check-assembly-accession.txt
              else
                  mv check-assembly-accession.txt F.1.Colony2-assembly-accession-error.txt
                  exit
              fi
          fi
      elif [ "paired-end" == "assembly" ]; then
          if [ "false" == "true" ]; then
              gzip -cd EMPTY_EXTRA > F.1.Colony2-art.fna
          else 
              cat EMPTY_EXTRA > F.1.Colony2-art.fna
          fi
      fi

      # Simulate reads from assembly, reads are 250bp without errors
      art_illumina -p -ss MSv3 -l 250 -m 400 -s 30 --fcov 150 -ir 0 -ir2 0 -dr 0 -dr2 0 -rs 42                         -na -qL 33 -qU 40 -o F.1.Colony2_R --id F.1.Colony2 -i F.1.Colony2-art.fna

      mv F.1.Colony2_R1.fq fastqs/F.1.Colony2_R1.fastq
      mv F.1.Colony2_R2.fq fastqs/F.1.Colony2_R2.fastq
      pigz -p 4 --fast fastqs/*.fastq
      cp F.1.Colony2-art.fna extra/F.1.Colony2.fna
      pigz -p 4 --best extra/F.1.Colony2.fna
  fi

  # Validate input FASTQs
  if [ "false" == "false" ]; then
      ERROR=0
      # Check paired-end reads have same read counts
      OPTS="--sample F.1.Colony2 --min_basepairs 2241820 --min_reads 7472 --min_proportion 0.5"
      if [ -f  "fastqs/F.1.Colony2_R2.fastq.gz" ]; then
          # Paired-end
          gzip -cd fastqs/F.1.Colony2_R1.fastq.gz | fastq-scan > r1.json
          gzip -cd fastqs/F.1.Colony2_R2.fastq.gz | fastq-scan > r2.json
          if ! reformat.sh in1=fastqs/F.1.Colony2_R1.fastq.gz in2=fastqs/F.1.Colony2_R2.fastq.gz qin=auto out=/dev/null 2> F.1.Colony2-paired-end-error.txt; then
              ERROR=1
              echo "F.1.Colony2 FASTQs contains an error. Please check the input FASTQs.
                  Further analysis is discontinued." |                 sed 's/^\s*//' >> F.1.Colony2-paired-end-error.txt
          else
              rm -f F.1.Colony2-paired-end-error.txt
          fi

          if ! check-fastqs.py --fq1 r1.json --fq2 r2.json ${OPTS}; then
              ERROR=1
          fi
          rm r1.json r2.json
      else
          # Single-end
          gzip -cd fastqs/F.1.Colony2.fastq.gz | fastq-scan > r1.json
          if ! check-fastqs.py --fq1 r1.json ${OPTS}; then
              ERROR=1
          fi
          rm r1.json
      fi

      # Failed validations so, let's keep them from continuing
      if [ "${ERROR}" -eq "1" ]; then
          mv fastqs/ failed-tests-fastqs/
      fi
  fi

  # Estimate Genome Size
  GENOME_SIZE_OUTPUT="F.1.Colony2-genome-size.txt"
  if [ "0" == "0" ]; then
      if [ "false" == "true" ]; then
          # Use the total assembly size as the genome size
          stats.sh in=extra/F.1.Colony2.fna.gz | grep All | awk '{print $5}' | sed 's/,//g' > ${GENOME_SIZE_OUTPUT}
      else
          FASTQS=""
          if [ -f  "fastqs/F.1.Colony2_R2.fastq.gz" ]; then
              FASTQS="-r fastqs/F.1.Colony2_R1.fastq.gz fastqs/F.1.Colony2_R2.fastq.gz"
          else
              FASTQS="fastqs/F.1.Colony2.fastq.gz"
          fi

          # First Pass
          mash sketch -o test -k 31 -m 3 ${FASTQS} 2>&1 |                 grep "Estimated genome size:" |                 awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
          rm -rf test.msh
          ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`

          # Check if second pass is needed
          if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ] || [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
              # Probably high coverage, try increasing number of kmer copies to 10
              M="-m 10"
              if [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
                  # Probably low coverage, try decreasing the number of kmer copies to 1
                  M="-m 1"
              fi
              mash sketch -o test -k 31 ${M} ${FASTQS} 2>&1 |                     grep "Estimated genome size:" |                     awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
              rm -rf test.msh
          fi
      fi

      # Check final estimate
      ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`
      if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ]; then
          rm ${GENOME_SIZE_OUTPUT}
          echo "F.1.Colony2 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) exceeds the maximum
                  allowed genome size (18040666 bp). If this is unexpected, please
                  investigate F.1.Colony2 to determine a cause (e.g. metagenomic, contaminants, etc...).
                  Otherwise, adjust the --max_genome_size parameter to fit your need. Further analysis
                  of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-genome-size-error.txt
      elif [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
          rm ${GENOME_SIZE_OUTPUT}
          echo "F.1.Colony2 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) is less than the minimum
                  allowed genome size (100000 bp). If this is unexpected, please
                  investigate F.1.Colony2 to determine a cause (e.g. metagenomic, contaminants, etc...).
                  Otherwise, adjust the --min_genome_size parameter to fit your need. Further analysis
                  of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-genome-size-error.txt
      fi
  else
executor >  local (63)
[fd/aca744] process > BACTOPIA:GATHER_SAMPLES (F.6.Colony7)  [ 14%] 63 of 421, failed: 63, retries: 62
[-        ] process > BACTOPIA:QC_READS                      -
[-        ] process > BACTOPIA:ASSEMBLE_GENOME               -
[-        ] process > BACTOPIA:ASSEMBLY_QC                   -
[-        ] process > BACTOPIA:ANNOTATE_GENOME               -
[-        ] process > BACTOPIA:MINMER_SKETCH                 -
[-        ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE      -
[-        ] process > BACTOPIA:ARIBA_ANALYSIS                -
[-        ] process > BACTOPIA:MINMER_QUERY                  -
[-        ] process > BACTOPIA:BLAST                         -
[-        ] process > BACTOPIA:CALL_VARIANTS                 -
[-        ] process > BACTOPIA:MAPPING_QUERY                 -
[-        ] process > BACTOPIA:SEQUENCE_TYPE                 -
[-        ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS   -
[7a/feb96a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[18/d6c833] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[e6/9d97b2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/81a4d9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[a1/110c50] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[52/28108b] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/81f9a4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[dd/6690ad] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[61/5a840d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ed/a3045d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[b1/907ab5] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/67a688] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d4/05ac13] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[9c/882aed] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[64/0f45f3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[47/b55766] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.10.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[ab/6e71a3] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[6a/6c3a49] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[cd/33e26f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[6d/53e37a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[5e/291202] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[c4/7984bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[46/5298ca] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6c/9f245a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.11.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fc/5d5eaa] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[ef/281a9e] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[7d/ab2b18] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[4a/befbf7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f7/561d56] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[14/0375ae] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[84/c01490] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[a9/b3919d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.12.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[78/11f751] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[f4/587355] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[d5/cf170a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[1e/b507bc] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[89/9e885d] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[dc/51e1ec] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[28/c55ac9] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[d7/0d284a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[2f/d1a6c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.3.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[f8/a288c4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[45/39f83a] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[fe/4ce8f2] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[3f/a0c7dd] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[4c/b2f2f8] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[2c/8de607] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[56/977539] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[40/bd49e4] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.4.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[a3/4816ba] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[ca/16d950] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[14/4ade94] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/d8309f] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[12/672031] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony6)` terminated with an error exit status (127) -- Execution is retried (1)
[41/b66597] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony7)` terminated with an error exit status (127) -- Execution is retried (1)
[fa/705527] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.5.Colony8)` terminated with an error exit status (127) -- Execution is retried (1)
[6e/024701] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony1)` terminated with an error exit status (127) -- Execution is retried (1)
[87/dccc87] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony2)` terminated with an error exit status (127) -- Execution is retried (1)
[b0/ae042c] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony3)` terminated with an error exit status (127) -- Execution is retried (1)
[7c/7c1ea0] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony4)` terminated with an error exit status (127) -- Execution is retried (1)
[e9/101dd7] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.6.Colony5)` terminated with an error exit status (127) -- Execution is retried (1)
[f0/f71411] NOTE: Process `BACTOPIA:GATHER_SAMPLES (F.1.Colony2)` terminated with an error exit status (127) -- Execution is retried (2)
Error executing process > 'BACTOPIA:GATHER_SAMPLES (F.1.Colony2)'

Caused by:
  Process requirement exceed available memory -- req: 6 GB; avail: 5.6 GB

Command executed:

  MERGED="multiple-read-sets-merged.txt"
  mkdir -p fastqs
  mkdir -p extra

  if [ "paired-end" == "paired-end" ]; then
      # Paired-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2_R1.fastq.gz
      cp -L 001-r2 fastqs/F.1.Colony2_R2.fastq.gz
      touch extra/empty.fna.gz
  elif [ "paired-end" == "single-end" ]; then
      # Single-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2.fastq.gz
      touch extra/empty.fna.gz
  elif [ "paired-end" == "ont" ]; then
      # Nanopore reads
      cp -L 001-r1 fastqs/F.1.Colony2.fastq.gz
      touch extra/empty.fna.gz
  elif  [ "paired-end" == "hybrid" ]; then
      # Paired-End Reads
      cp -L 001-r1 fastqs/F.1.Colony2_R1.fastq.gz
      cp -L 001-r2 fastqs/F.1.Colony2_R2.fastq.gz
      cp -L EMPTY_EXTRA extra/F.1.Colony2.fastq.gz
  elif [ "paired-end" == "merge-pe" ] || [ "paired-end" == "hybrid-merge-pe" ]; then 
      # Merge Paired-End Reads
      echo "This sample had reads merged." > ${MERGED}
      echo "R1:" >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2_R1.fastq.gz
      echo "Merged R1:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2_R1.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

      echo "R2:" >> ${MERGED}
      find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r2" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2_R2.fastq.gz
      echo "Merged R2:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2_R2.fastq.gz | awk '{print $5"    "$9}' >> ${MERGED}

      if [ "paired-end" == "hybrid-merge-pe" ]; then
          cp -L EMPTY_EXTRA extra/F.1.Colony2.fastq.gz
      else
          touch extra/empty.fna.gz
      fi
  elif [ "paired-end" == "merge-se" ]; then 
      # Merge Single-End Reads
      echo "This sample had reads merged." > ${MERGED}
      echo "SE:" >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} ls -l {} | awk '{print $5"    "$9}' >> ${MERGED}
      find -name "*r1" | sort | xargs -I {} readlink {} | xargs -I {} cat {} > fastqs/F.1.Colony2.fastq.gz
      echo "Merged SE:" >> ${MERGED}
      ls -l fastqs/F.1.Colony2.fastq.gz | awk '{print $5"   "$9}' >> ${MERGED}

      touch extra/empty.fna.gz
  elif [ "paired-end" == "sra_accession" ]; then
      if [ "3" == "3" ]; then
          echo "Unable to download F.1.Colony2 from both SRA and ENA 3 times. This may or may 
              not be a temporary connection issue. Rather than stop the whole Bactopia run, 
              further analysis of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-fastq-download-error.txt
          exit
      else
          # Download accession from ENA/SRA
          fastq-dl F.1.Colony2 SRA                 --cpus 4                 --outdir fastqs/                 --group_by_experiment                 --is_experiment                 --ftp_only
          touch extra/empty.fna.gz
      fi 
  elif [ "false" == "true" ]; then
      if [ "paired-end" == "assembly_accession" ]; then
          if [ "3" == "3" ]; then
              touch extra/empty.fna.gz
              echo "Unable to download F.1.Colony2 from NCBI Assembly 3 times. This may or may
                  not be a temporary connection issue. Rather than stop the whole Bactopia run, 
                  further analysis of F.1.Colony2 will be discontinued." |                 sed 's/^\s*//' > F.1.Colony2-assembly-download-error.txt
              exit
          else
              # Verify Assembly accession
              check-assembly-accession.py F.1.Colony2 > accession.txt 2> check-assembly-accession.txt

              if [ -s "accession.txt" ]; then
                  # Download from NCBI assembly and simulate reads
                  mkdir fasta/
                  ncbi-genome-download bacteria -o ./ -F fasta -p 4                                                 -s null -A accession.txt -r 50 
                  find . -name "*F.1.Colony2*.fna.gz" | xargs -I {} mv {} fasta/
                  rename 's/(GC[AF]_\d+).*/$1.fna.gz/' fasta/*
                  gzip -cd fasta/F.1.Colony2.fna.gz > F.1.Colony2-art.fna
                  rm check-assembly-accession.txt
              else
                  mv check-assembly-accession.txt F.1.Colony2-assembly-accession-error.txt
                  exit
              fi
          fi
      elif [ "paired-end" == "assembly" ]; then
          if [ "false" == "true" ]; then
              gzip -cd EMPTY_EXTRA > F.1.Colony2-art.fna
          else 
              cat EMPTY_EXTRA > F.1.Colony2-art.fna
          fi
      fi

      # Simulate reads from assembly, reads are 250bp without errors
      art_illumina -p -ss MSv3 -l 250 -m 400 -s 30 --fcov 150 -ir 0 -ir2 0 -dr 0 -dr2 0 -rs 42                         -na -qL 33 -qU 40 -o F.1.Colony2_R --id F.1.Colony2 -i F.1.Colony2-art.fna

      mv F.1.Colony2_R1.fq fastqs/F.1.Colony2_R1.fastq
      mv F.1.Colony2_R2.fq fastqs/F.1.Colony2_R2.fastq
      pigz -p 4 --fast fastqs/*.fastq
      cp F.1.Colony2-art.fna extra/F.1.Colony2.fna
      pigz -p 4 --best extra/F.1.Colony2.fna
  fi

  # Validate input FASTQs
  if [ "false" == "false" ]; then
      ERROR=0
      # Check paired-end reads have same read counts
      OPTS="--sample F.1.Colony2 --min_basepairs 2241820 --min_reads 7472 --min_proportion 0.5"
      if [ -f  "fastqs/F.1.Colony2_R2.fastq.gz" ]; then
          # Paired-end
          gzip -cd fastqs/F.1.Colony2_R1.fastq.gz | fastq-scan > r1.json
          gzip -cd fastqs/F.1.Colony2_R2.fastq.gz | fastq-scan > r2.json
          if ! reformat.sh in1=fastqs/F.1.Colony2_R1.fastq.gz in2=fastqs/F.1.Colony2_R2.fastq.gz qin=auto out=/dev/null 2> F.1.Colony2-paired-end-error.txt; then
              ERROR=1
              echo "F.1.Colony2 FASTQs contains an error. Please check the input FASTQs.
                  Further analysis is discontinued." |                 sed 's/^\s*//' >> F.1.Colony2-paired-end-error.txt
          else
              rm -f F.1.Colony2-paired-end-error.txt
          fi

          if ! check-fastqs.py --fq1 r1.json --fq2 r2.json ${OPTS}; then
              ERROR=1
          fi
          rm r1.json r2.json
      else
          # Single-end
          gzip -cd fastqs/F.1.Colony2.fastq.gz | fastq-scan > r1.json
          if ! check-fastqs.py --fq1 r1.json ${OPTS}; then
              ERROR=1
          fi
          rm r1.json
      fi

      # Failed validations so, let's keep them from continuing
      if [ "${ERROR}" -eq "1" ]; then
          mv fastqs/ failed-tests-fastqs/
      fi
  fi

  # Estimate Genome Size
  GENOME_SIZE_OUTPUT="F.1.Colony2-genome-size.txt"
  if [ "0" == "0" ]; then
      if [ "false" == "true" ]; then
          # Use the total assembly size as the genome size
          stats.sh in=extra/F.1.Colony2.fna.gz | grep All | awk '{print $5}' | sed 's/,//g' > ${GENOME_SIZE_OUTPUT}
      else
          FASTQS=""
          if [ -f  "fastqs/F.1.Colony2_R2.fastq.gz" ]; then
              FASTQS="-r fastqs/F.1.Colony2_R1.fastq.gz fastqs/F.1.Colony2_R2.fastq.gz"
          else
              FASTQS="fastqs/F.1.Colony2.fastq.gz"
          fi

          # First Pass
          mash sketch -o test -k 31 -m 3 ${FASTQS} 2>&1 |                 grep "Estimated genome size:" |                 awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
          rm -rf test.msh
          ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`

          # Check if second pass is needed
          if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ] || [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
              # Probably high coverage, try increasing number of kmer copies to 10
              M="-m 10"
              if [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
                  # Probably low coverage, try decreasing the number of kmer copies to 1
                  M="-m 1"
              fi
              mash sketch -o test -k 31 ${M} ${FASTQS} 2>&1 |                     grep "Estimated genome size:" |                     awk '{if($4){printf("%d\n", $4)}} END {if (!NR) print "0"}' > ${GENOME_SIZE_OUTPUT}
              rm -rf test.msh
          fi
      fi

      # Check final estimate
      ESTIMATED_GENOME_SIZE=`head -n1 ${GENOME_SIZE_OUTPUT}`
      if [ ${ESTIMATED_GENOME_SIZE} -gt "18040666" ]; then
          rm ${GENOME_SIZE_OUTPUT}
          echo "F.1.Colony2 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) exceeds the maximum
                  allowed genome size (18040666 bp). If this is unexpected, please
                  investigate F.1.Colony2 to determine a cause (e.g. metagenomic, contaminants, etc...).
                  Otherwise, adjust the --max_genome_size parameter to fit your need. Further analysis
                  of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-genome-size-error.txt
      elif [ ${ESTIMATED_GENOME_SIZE} -lt "100000" ]; then
          rm ${GENOME_SIZE_OUTPUT}
          echo "F.1.Colony2 estimated genome size (${ESTIMATED_GENOME_SIZE} bp) is less than the minimum
                  allowed genome size (100000 bp). If this is unexpected, please
                  investigate F.1.Colony2 to determine a cause (e.g. metagenomic, contaminants, etc...).
                  Otherwise, adjust the --min_genome_size parameter to fit your need. Further analysis
                  of F.1.Colony2 will be discontinued." |             sed 's/^\s*//' > F.1.Colony2-genome-size-error.txt
      fi
  else
      # Use the genome size given by the user. (Should be >= 0)
      echo "0" > ${GENOME_SIZE_OUTPUT}
  fi

  # Capture versions
  cat <<-END_VERSIONS > versions.yml
  "BACTOPIA:GATHER_SAMPLES":
      art: $(echo $(art_illumina --help 2>&1) | sed 's/^.*Version //;s/ .*$//')
      fastq-dl: $(echo $(fastq-dl --version 2>&1) | sed 's/fastq-dl //')
      fastq-scan: $(echo $(fastq-scan -v 2>&1) | sed 's/fastq-scan //')
      mash: $(echo $(mash --version 2>&1))
      ncbi-genome-download: $(echo $(ncbi-genome-download --version 2>&1))
      pigz: $(echo $(pigz --version 2>&1) | sed 's/pigz //')
  END_VERSIONS

Command exit status:
  -

Command output:
  (empty)

Command error:
  env: ‘singularity’: No such file or directory

Work dir:
  /scr1/users/theillere/klebsiella-bactopia/work/cb/5dd50e99359f9b9354d2c725952f7f

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

(bactopia) [theillere@reslnvvhpc0153 klebsiella-bactopia]$
rpetit3 commented 2 years ago

hmmm for some reason it went back to local executor and not slurm.

I think we'll need to tweak you slurm config file a little

rpetit3 commented 2 years ago

Before we tweak your config, I wonder what happens with -profile slurm,respublica

erin-thei commented 2 years ago
Screen Shot 2022-07-30 at 9 31 41 AM Screen Shot 2022-07-30 at 8 50 45 AM

Trying on one sample, I ran the command: bactopia --R1 I.351.2.Colony4_R1.fastq.gz --R2 I.351.2.Colony4_R2.fastq.gz --sample I.351.2.Colony4 --species 'Klebsiella pneumoniae' --datasets datasets/ --coverage 100 --nfconfig /mnt/isilon/marcg_genomics/bactopia_respublica.config --cleanup_workdir -profile slurm,respublica

It produced an error but didn't terminate the process.

rpetit3 commented 2 years ago

I can't tell from this image if its going to SLURM or not. I think we're close, good news is once we get your profile figured out, it will just work.

Remind me, was there a partition name that you submit jobs to? You can get this information with sinfo

erin-thei commented 2 years ago

It didn't go to SLURM. The only time it seems to have go to slurm was when using the -profile slurm, but that failed as stated in one of the above messages. I'm also not sure about the partition name that I submit jobs too, but here is a screenshot of the output of sinfo

Screen Shot 2022-08-02 at 5 41 30 PM
erin-thei commented 2 years ago

A colleague told me that he was able to successfully run Bactopia on the same cluster I am using by using -profile respublica,standard. When I tried that I ran into another error message: (it also seemed to not even go to slurm...)

(bactopia) [theillere@reslnvvhpc0153 klebsiella-bactopia]$ bactopia --R1 I.351.2.Colony4_R1.fastq.gz --R2 I.351.2.Colony4_R2.fastq.gz --sample I.351.2.Colony4 --species 'Klebsiella pneumoniae' --datasets datasets/ --coverage 100 --nfconfig /mnt/isilon/marc_genomics/bactopia_respublica.config -profile respublica,standard --cleanup_workdir N E X T F L O W ~ version 22.04.0 Launching /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/main.nf [naughty_yalow] DSL2 - revision: bd75c553f9

| | _ | | __ (_)
| ' \ / ` |/ | / | ' | |/ ` |
| |) | (| | (| || () | |) | | (| |
|./ _,|_|__/| ./||_,| |_|
bactopia v2.0.3 Bactopia is a flexible pipeline for complete analysis of bacterial genomes.

Core Nextflow options runName : naughty_yalow launchDir : /scr1/users/theillere/klebsiella-bactopia workDir : /scr1/users/theillere/klebsiella-bactopia/work projectDir : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x userName : theillere profile : respublica,standard configFiles : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/nextflow.config

Required Parameters R1 : I.351.2.Colony4_R1.fastq.gz R2 : I.351.2.Colony4_R2.fastq.gz sample : I.351.2.Colony4

Dataset Parameters datasets : datasets/ species : Klebsiella pneumoniae

Nextflow Configuration Parameters nfconfig : /mnt/isilon/marc_genomics/bactopia_respublica.config cleanup_workdir : true

Nextflow Profile Parameters condadir : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/conda/envs registry : quay singularity_cache: /home/theillere/.bactopia/singularity

!! Only displaying parameters that differ from the pipeline defaults !!

If you use bactopia for your analysis please cite:

If something looks wrong, now's your chance to back out (CTRL+C 3 times). Sleeping for 5 seconds...

executor > local (2) [b6/3b29f5] process > BACTOPIA:GATHER_SAMPLES (I.... [100%] 1 of 1 ✔ [69/9446d4] process > BACTOPIA:QC_READS (I.351.2.... [100%] 1 of 1 ✔ [- ] process > BACTOPIA:ASSEMBLE_GENOME - [- ] process > BACTOPIA:ASSEMBLY_QC - [- ] process > BACTOPIA:ANNOTATE_GENOME - [- ] process > BACTOPIA:MINMER_SKETCH [ 0%] 0 of 1 [- ] process > BACTOPIA:ANTIMICROBIAL_RESI... - [- ] process > BACTOPIA:ARIBA_ANALYSIS - [- ] process > BACTOPIA:MINMER_QUERY - [- ] process > BACTOPIA:BLAST - [- ] process > BACTOPIA:CALL_VARIANTS - [- ] process > BACTOPIA:MAPPING_QUERY - [- ] process > BACTOPIA:SEQUENCE_TYPE - [- ] process > BACTOPIA:CUSTOM_DUMPSOFTWAR... - Error executing process > 'BACTOPIA:MINMER_SKETCH (I.351.2.Colony4)'

Caused by: Process requirement exceed available memory -- req: 8 GB; avail: 5.6 GB

Command executed:

gzip -cd I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz | mash sketch -o I.351.2.Colony4-k21 -k 21 -s 10000 -r -I I.351.2.Colony4 - gzip -cd I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz | mash sketch -o I.351.2.Colony4-k31 -k 31 -s 10000 -r -I I.351.2.Colony4 - sourmash sketch dna -p k=21,k=31,k=51,abund,scaled=10000 --merge I.351.2.Colony4 -o I.351.2.Colony4.sig I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz

if [[ "false" == "true" ]]; then mccortex31 build -f -k 31 -s I.351.2.Colony4 -2 I.351.2.Colony4_R1.fastq.gz:I.351.2.Colony4_R2.fastq.gz -t 4 -m 7500mb -q temp_counts if [ "false" == "false" ]; then

Clean up Cortex file (mostly remove singletons)

      mccortex31 clean -q -B 2 -U2 -T2 -m 7500mb -o I.351.2.Colony4.ctx temp_counts
      rm temp_counts
  else
      mv temp_counts I.351.2.Colony4.ctx
  fi

fi

Capture versions

cat <<-END_VERSIONS > versions.yml "BACTOPIA:MINMER_SKETCH": mash: $(echo $(mash 2>&1) | sed 's/^.Mash version //;s/ .$//') mccortex: $(echo $(mccortex31 2>&1) | sed 's/^.mccortex=v//;s/ .$//') sourmash: $(echo $(sourmash --version 2>&1) | sed 's/sourmash //;') END_VERSIONS

Command exit status:

Command output: (empty)

Work dir: /scr1/users/theillere/klebsiella-bactopia/work/c6/daa090c89597d0b3e747260f2cc9f9

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

Bactopia Execution Summary
---------------------------
Bactopia Version : 2.0.3
Nextflow Version : 22.04.0
Command Line     : nextflow run /home/theillere/miniconda3/envs/bactopia//share/bactopia-2.0.x/main.nf -w /scr1/users/theillere/klebsiella-bactopia/work/ --R1 I.351.2.Colony4_R1.fastq.gz --R2 I.351.2.Colony4_R2.fastq.gz --sample I.351.2.Colony4 --species 'Klebsiella pneumoniae' --datasets datasets/ --coverage 100 --nfconfig /mnt/isilon/marc_genomics/bactopia_respublica.config -profile respublica,standard --cleanup_workdir
Resumed          : false
Completed At     : 2022-08-01T19:39:06.015692-04:00
Duration         : 10m 39s
Success          : false
Exit Code        : null
Error Report     : Error executing process > 'BACTOPIA:MINMER_SKETCH (I.351.2.Colony4)'

Caused by: Process requirement exceed available memory -- req: 8 GB; avail: 5.6 GB

Command executed:

gzip -cd I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz | mash sketch -o I.351.2.Colony4-k21 -k 21 -s 10000 -r -I I.351.2.Colony4 - gzip -cd I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz | mash sketch -o I.351.2.Colony4-k31 -k 31 -s 10000 -r -I I.351.2.Colony4 - sourmash sketch dna -p k=21,k=31,k=51,abund,scaled=10000 --merge I.351.2.Colony4 -o I.351.2.Colony4.sig I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz

if [[ "false" == "true" ]]; then mccortex31 build -f -k 31 -s I.351.2.Colony4 -2 I.351.2.Colony4_R1.fastq.gz:I.351.2.Colony4_R2.fastq.gz -t 4 -m 7500mb -q temp_counts if [ "false" == "false" ]; then

Clean up Cortex file (mostly remove singletons)

      mccortex31 clean -q -B 2 -U2 -T2 -m 7500mb -o I.351.2.Colony4.ctx temp_counts
      rm temp_counts
  else
      mv temp_counts I.351.2.Colony4.ctx
  fi

fi

executor > local (2) [b6/3b29f5] process > BACTOPIA:GATHER_SAMPLES (I.... [100%] 1 of 1 ✔ [69/9446d4] process > BACTOPIA:QC_READS (I.351.2.... [100%] 1 of 1 ✔ [- ] process > BACTOPIA:ASSEMBLE_GENOME - [- ] process > BACTOPIA:ASSEMBLY_QC - [- ] process > BACTOPIA:ANNOTATE_GENOME - [c6/daa090] process > BACTOPIA:MINMER_SKETCH (I.3... [100%] 1 of 1, failed: 1 [- ] process > BACTOPIA:ANTIMICROBIAL_RESI... - [- ] process > BACTOPIA:ARIBA_ANALYSIS [ 0%] 0 of 2 [- ] process > BACTOPIA:MINMER_QUERY - [- ] process > BACTOPIA:BLAST - [- ] process > BACTOPIA:CALL_VARIANTS - [- ] process > BACTOPIA:MAPPING_QUERY - [- ] process > BACTOPIA:SEQUENCE_TYPE - [- ] process > BACTOPIA:CUSTOM_DUMPSOFTWAR... - Error executing process > 'BACTOPIA:MINMER_SKETCH (I.351.2.Colony4)'

Caused by: Process requirement exceed available memory -- req: 8 GB; avail: 5.6 GB

Command executed:

gzip -cd I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz | mash sketch -o I.351.2.Colony4-k21 -k 21 -s 10000 -r -I I.351.2.Colony4 - gzip -cd I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz | mash sketch -o I.351.2.Colony4-k31 -k 31 -s 10000 -r -I I.351.2.Colony4 - sourmash sketch dna -p k=21,k=31,k=51,abund,scaled=10000 --merge I.351.2.Colony4 -o I.351.2.Colony4.sig I.351.2.Colony4_R1.fastq.gz I.351.2.Colony4_R2.fastq.gz

if [[ "false" == "true" ]]; then mccortex31 build -f -k 31 -s I.351.2.Colony4 -2 I.351.2.Colony4_R1.fastq.gz:I.351.2.Colony4_R2.fastq.gz -t 4 -m 7500mb -q temp_counts if [ "false" == "false" ]; then

Clean up Cortex file (mostly remove singletons)

      mccortex31 clean -q -B 2 -U2 -T2 -m 7500mb -o I.351.2.Colony4.ctx temp_counts
      rm temp_counts
  else
      mv temp_counts I.351.2.Colony4.ctx
  fi

fi

Capture versions

cat <<-END_VERSIONS > versions.yml "BACTOPIA:MINMER_SKETCH": mash: $(echo $(mash 2>&1) | sed 's/^.Mash version //;s/ .$//') mccortex: $(echo $(mccortex31 2>&1) | sed 's/^.mccortex=v//;s/ .$//') sourmash: $(echo $(sourmash --version 2>&1) | sed 's/sourmash //;') END_VERSIONS

Command exit status:

Command output: (empty)

Work dir: /scr1/users/theillere/klebsiella-bactopia/work/c6/daa090c89597d0b3e747260f2cc9f9

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

(bactopia) [theillere@reslnvvhpc0153 klebsiella-bactopia]$ cd work/c6/daa090c89597d0b3e747260f2cc9f9/

rpetit3 commented 2 years ago

Let's try this! (Also thank you very much for continuing to try things)

-profile slurm --slurm_queue defq -qs 20

We're going to try using the built in profile for SLURM, but tell it to use the defq partition (taken from sinfo), and limit the queue size to 20 (-qs 20)

erin-thei commented 2 years ago

Thank you for all of the help! Question before I try this, should I still be using my config file when running these options?

rpetit3 commented 2 years ago

You should not need your config.

erin-thei commented 2 years ago

Sorry for the delay. I tried using those parameters but still got an error message. This is the command I tried: (bactopia) [theillere@reslnvvhpc097 klebsiella-bactopia]$ bactopia --R1 I.350.2.Colony9_R1.fastq.gz --R2 I.350.2.Colony9_R2.fastq.gz --sample I.350.2.Colony9 --datasets datasets/ --species 'Klebsiella pneumoniae' --coverage 100 -profile slurm --slurm_queue defq -qs 20 --cleanup_workdir

It looks like it went to SLURM but it did not get past the gather samples step. I killed the command after it retried to gather the samples 10 times.

(bactopia) [theillere@reslnvvhpc097 klebsiella-bactopia]$ bactopia --R1 I.350.2.Colony9_R1.fastq.gz --R2 I.350.2.Colony9_R2.fastq.gz --sample I.350.2.Colony9 --datasets datasets/ --species 'Klebsiella pneumoniae' --coverage 100 -profile slurm --slurm_queue defq -qs 20 --cleanup_workdir 2022-08-02 20:53:34:root:INFO - Checking if environment pre-builds are needed, use --verbose to see full details. 2022-08-02 20:53:34:root:INFO - Found annotate_genome (1 of 7), begin build to /home/theillere/.bactopia/singularity/quay.io-bactopia-annotate_genome-2.0.3.img External command isn't available!

Command: bash -c 'singularity build /home/theillere/.bactopia/singularity/quay.io-bactopia-annotate_genome-2.0.3.img docker://quay.io/bactopia/annotate_genome:2.0.3'

Search path: ['/cm/local/apps/environment-modules/4.5.3/bin', '/home/theillere/miniconda3/envs/bactopia/bin', '/home/theillere/miniconda3/condabin', '/home/theillere/.local/bin', '/home/theillere/bin', '/cm/shared/apps/slurm/current/sbin', '/cm/shared/apps/slurm/current/bin', '/usr/local/bin', '/usr/local/sbin', '/usr/bin', '/usr/sbin', '/sbin', '/opt/puppetlabs/bin']

Standard error: bash: singularity: command not found N E X T F L O W ~ version 22.04.0 Launching /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/main.nf [lonely_celsius] DSL2 - revision: bd75c553f9

| | _ | | __ (_)
| ' \ / ` |/ | / | ' | |/ ` |
| |) | (| | (| || () | |) | | (| |
|./ _,|_|__/| ./||_,| |_|
bactopia v2.0.3 Bactopia is a flexible pipeline for complete analysis of bacterial genomes.

Core Nextflow options runName : lonely_celsius containerEngine : singularity container : quay.io/bactopia/bactopia:2.0.3 launchDir : /scr1/users/theillere/klebsiella-bactopia workDir : /scr1/users/theillere/klebsiella-bactopia/work projectDir : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x userName : theillere profile : slurm configFiles : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/nextflow.config

Required Parameters R1 : I.350.2.Colony9_R1.fastq.gz R2 : I.350.2.Colony9_R2.fastq.gz sample : I.350.2.Colony9

Dataset Parameters datasets : datasets/ species : Klebsiella pneumoniae

SLURM Profile (-profile slurm) Parameters slurm_queue : defq

Nextflow Configuration Parameters cleanup_workdir : true

Nextflow Profile Parameters condadir : /home/theillere/miniconda3/envs/bactopia/share/bactopia-2.0.x/conda/envs registry : quay singularity_cache: /home/theillere/.bactopia/singularity

!! Only displaying parameters that differ from the pipeline defaults !!

If you use bactopia for your analysis please cite:

If something looks wrong, now's your chance to back out (CTRL+C 3 times). Sleeping for 5 seconds...

executor > slurm (2) [b9/eeb2cd] process > BACTOPIA:GATHER_SAMPLES (I.350.2.Colony9) [ 50%] 1 of 2, failed: 1, retries: 1 [- ] process > BACTOPIA:QC_READS - [- ] process > BACTOPIA:ASSEMBLE_GENOME - [- ] process > BACTOPIA:ASSEMBLY_QC - [- ] process > BACTOPIA:ANNOTATE_GENOME - [- ] process > BACTOPIA:MINMER_SKETCH - [- ] process > BACTOPIA:ANTIMICROBIAL_RESISTANCE - [- ] process > BACTOPIA:ARIBA_ANALYSIS - [- ] process > BACTOPIA:MINMER_QUERY - [- ] process > BACTOPIA:BLAST - [- ] process > BACTOPIA:CALL_VARIANTS - [- ] process > BACTOPIA:MAPPING_QUERY - [- ] process > BACTOPIA:SEQUENCE_TYPE - [- ] process > BACTOPIA:CUSTOM_DUMPSOFTWAREVERSIONS - [97/3735cb] NOTE: Process BACTOPIA:GATHER_SAMPLES (I.350.2.Colony9) terminated with an error exit status (127) -- Execution is retried (1)

rpetit3 commented 2 years ago

Here's our issue

bash: singularity: command not found

Do you know if you have to do something like module load singularity on your cluster?

erin-thei commented 2 years ago

Yes, we have module load singularity. So, how would the command look using that?

rpetit3 commented 2 years ago

I think try loading the singularity module then running the bactopia command

erin-thei commented 2 years ago

And then run the same command (i.e. with -profile slurm --slurm_queue defq -qs 20)?

rpetit3 commented 2 years ago

yep

erin-thei commented 2 years ago
Screen Shot 2022-08-04 at 3 24 10 PM Screen Shot 2022-08-04 at 3 24 28 PM Screen Shot 2022-08-04 at 3 24 42 PM

Still running into issues running it even when using module load singularity. Also looks like there's limited space in tmp folder. Not sure where to go from here. Appreciate you bearing through this with me! Is there at some point where I should consider re-installing bactopia since I'm having issues even running multiple samples on my local computer.

rpetit3 commented 2 years ago

Does your cluster have a certain folder with more space they want you to use?

erin-thei commented 2 years ago

i've been told that when working on a cluster, the tmp dir is limited in space, so is there an option to tell nextflow to use the current working directory as a tmp dir? (sorry if this is a simple question, just trying to understand a bit more -- thanks!!)

rpetit3 commented 2 years ago

I think you can use NXF_TEMP to change the temporary directory used by Nextflow

rpetit3 commented 5 months ago

Hi @erin-thei

This issue was a lengthy one! But, I'm now cleaning up old issues and since this is related to v2, I'm going to go ahead and close this with the recommendation to give v3 a try.

Please reach out if you have any questions or issues!

Cheers, Robert