MaestSi / MetONTIIME

A Meta-barcoding pipeline for analysing ONT data in QIIME2 framework
GNU General Public License v3.0
78 stars 17 forks source link

Evaluate_diversity.sh Error #24

Closed timregan closed 3 years ago

timregan commented 3 years ago

Most of the time, the Evaluate_diversity.sh script works perfectly for me. However, I am receiving a recurring error on some slightly larger datasets (e.g. 20 samples, subsampled at 20k reads). The script seems to run ok until the "qiime phylogeny align-to-tree-mafft-fasttree" command when I receive the following error:

Plugin error from phylogeny: Command '['mafft', '--preservecase', '--inputorder', '--thread', '8', '/tmp/qiime2-archive-so3u883h/9152bd3d-d8d5-472b-9c98-8c868156dfa3/data/dna-sequences.fasta']' returned non-zero exit status 1.

Debug info has been saved to /tmp/qiime2-q2cli-err-vund61s9.log

$ head -50 /tmp/qiime2-q2cli-err-vund61s9.log

Running external command line application. This may print messages to stdout and/or stderr. The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: mafft --preservecase --inputorder --thread 8 /tmp/qiime2-archive-so3u883h/9152bd3d-d8d5-472b-9c98-8c868156dfa3/data/dna-sequences.fasta

Traceback (most recent call last): File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/q2cli/commands.py", line 329, in call results = action(arguments) File "", line 2, in align_to_tree_mafft_fasttree File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable output_types, provenance) File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/qiime2/sdk/action.py", line 484, in _callableexecutor outputs = self._callable(scope.ctx, view_args) File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/q2_phylogeny/_align_to_tree_mafft_fasttree.py", line 20, in align_to_tree_mafft_fasttree parttree=parttree) File "", line 2, in mafft File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable output_types, provenance) File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callableexecutor output_views = self._callable(**view_args) File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/q2_alignment/_mafft.py", line 127, in mafft return _mafft(sequences_fp, None, n_threads, parttree) File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/q2_alignment/_mafft.py", line 99, in _mafft run_command(cmd, result_fp) File "/anaconda/envs/MetONTIIME_env/lib/python3.6/site-packages/q2_alignment/_mafft.py", line 26, in run_command subprocess.run(cmd, stdout=output_f, check=True) File "/anaconda/envs/MetONTIIME_env/lib/python3.6/subprocess.py", line 438, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['mafft', '--preservecase', '--inputorder', '--thread', '8', '/tmp/qiime2-archive-so3u883h/9152bd3d-d8d5-472b-9c98-8c868156dfa3/data/dna-sequences.fasta']' returned non-zero exit status 1.

I've tried increasing the number of cores to 12, but the same error appears. Do you think it's RAM or something else? Many thanks, Tim

MaestSi commented 3 years ago

Hi, I am not sure but probably yes, it may be due to the RAM, if it happens only when considering a greater amount of reads (across all samples). You may try either reducing the number of reads or the identity threshold for clustering (this will reduce the number of representative sequences used for building the phylogenetic tree)...or, in alternative, you may use Evaluate_diversity_non_phylogenetic.sh script. Let me know if you are able to solve it! Simone

MaestSi commented 3 years ago

Hi Tim, I will close the issue for now. In case you have any other problems, feel free to reopen it. Simone

timregan commented 3 years ago

Hi Simone, Apologies for the delayed response, our University server was down for a whole week! Anyway, it looks like RAM was the issue. I tried it again with > 128G RAM and that seems to have worked. One set of data is still freezing, but I'll request more RAM next time for that and I'm sure it will work too. Thanks again, Tim