Workflow is failing at diversity step for some users

[lukenoaa]

Error message from @sandenoaa:

Error in library(msa) : there is no package called ‘msa’
Execution halted
cat: temp_odseq: No such file or directory
rm: temp_odseq: No such file or directory
Error in job alignment_detect_outliers while creating output file 02-output-dada2-pe-unfiltered/02-alignment-tree/aligned_repseqs_outliers.tsv.
RuleException:
CalledProcessError in line 967 of /Users/seananderson/Gomecc-4/18S/tourmaline-18S-01312023/Snakefile:
Command 'Rscript --vanilla scripts/run_odseq.R 02-output-dada2-pe-unfiltered/02-alignment-tree/aligned_repseqs.fasta linear 100 0.025 temp_odseq; cat temp_odseq | sed 's/^X//' > 02-output-dada2-pe-unfiltered/02-alignment-tree/aligned_repseqs_outliers.tsv; /bin/rm temp_odseq' returned non-zero exit status 1.
  File "/Users/seananderson/Gomecc-4/18S/tourmaline-18S-01312023/Snakefile", line 967, in __rule_alignment_detect_outliers
  File "/Users/seananderson/miniconda3/envs/qiime2-2021.2/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Removing output files of failed job alignment_detect_outliers since they might be corrupted:
02-output-dada2-pe-unfiltered/02-alignment-tree/aligned_repseqs_outliers.tsv
Will exit after finishing currently running jobs.
Exiting because a job execution failed. Look above for error message

Options to fix:

• give the option not to run odseq/msa
• delete those steps entirely
• find a python library that can do the same thing

[lukenoaa]

Note that this error could not be reproduced by @ksilnoaa or @rckarns8 running the workflow on cetacean microbiome and sedtrap datasets, respectively. Perhaps this issue is specific to @sandenoaa's installation/environment. Does it occur when using the Docker container also?

[rckarns8]

I was able to reproduce this bug, and will work through troubleshooting it. Note: on Intel Mac, not M1 chip, with a conda environment.

[ksilnoaa]

This error looks like it is due to msa not installing correctly. @rckarns8 can you check out your installed R packages list when running R from within the qiime2.2021 conda environment?

[rckarns8]

Yep, I re-installed msa and am re-running the dada2_pe_diversity_unfiltered now.

[rckarns8]

New Error:

202-output-dada2-pe-unfiltered/02-alignment-tree/aligned_repseqs.fasta linear 100 0.025 temp_odseq; cat temp_odseq | sed 's/^X//' > 02-output-dada2-pe-unfiltered/02-alignment-tree/aligned_repseqs_outliers.tsv; /bin/rm temp_odseq' returned non-zero exit status 1.

  File "/Users/rachaelkarns/Desktop/Tourmaline/tourmaline-GoMCpump/Snakefile", line 967, in __rule_alignment_detect_outliers

  File "/Users/rachaelkarns/miniconda3/envs/qiime2-2021.2/lib/python3.6/concurrent/futures/thread.py", line 56, in run

Removing output files of failed job alignment_detect_outliers since they might be corrupted:

02-output-dada2-pe-unfiltered/02-alignment-tree/aligned_repseqs_outliers.tsv

Will exit after finishing currently running jobs.

Exiting because a job execution failed. Look above for error message

• Installing Odseq fresh, in R under my conda environment, and re-initializing the job.

[rckarns8]

Compatibility issue- Odseq, when installed with bioconductor, requires kebabs, v. >1.4.1, but when installed as a dependency, Kebabs is installed as v. 1.24.0.

Potential Fix- manually install the correct version of kebabs, then reinstall odseq without dependencies, if needed.

[rckarns8]

Temporary Workaround:

Install the R package 'odseq' using conda, not the BiocManager plug in in the R environment.

conda install -c bioconda bioconductor-odseq

This fixed all dependency issues, as noted in the previous comment, and the diversity step completed with no errors. @sandenoaa , if you have a chance let me know if this workaround works for you as well.

[lukenoaa]

Great news. I will be interested to see if it's possible to install all of the dependencies (including these more troublesome ones) from an exported conda yaml file.

[lukenoaa]

This issue is resolved with new version running qiime2-2023.2.