MaestSi / MetONTIIME

A Meta-barcoding pipeline for analysing ONT data in QIIME2 framework
GNU General Public License v3.0
76 stars 17 forks source link

process killed in 'assign taxonomy' #71

Closed dloukovi closed 11 months ago

dloukovi commented 11 months ago

Dear Simone, can you help me with the following error? It does not make sense to crash...I am only running one fastq.gz file (50.000 reads) in an intel i9 with 100 GB RAM and 10 cores (20 threads)

Error executing process > 'assignTaxonomy (1)'

Caused by: Process assignTaxonomy (1) terminated with an error exit status (137)

Command executed:

mkdir -p /home/dimitrios/MetONTIIME/assignTaxonomy

classifier_uc=$(awk '{print toupper($0)'} <<< Vsearch)

if [ "$classifier_uc" == "BLAST" ]; then echo "Clqiime feature-classifier classify-consensus-blast ast and Vsearch)" --i-query /home/dimitrios/MetONTIIME/derepSeq/rep-seqs.qza --i-reference-reads /home/dimitrios/MetONTIIME/importDb/db_sequences.qza --i-reference-taxonomy /home/dimitrios/MetONTIIME/importDb/db_taxonomy.qza --p-perc-identity 0.9 --p-query-cov 0.8 --p-maxaccepts 3 --o-classification /home/dimitrios/MetONTIIME/assignTaxonomy/taxonomy.qza --o-search-results /home/dimitrios/MetONTIIME/assignTaxonomy/search_results.qza elif [ "$classifier_uc" == "VSEARCH" ]; then qiime taxa filter-tablqiime feature-classifier classify-consensus-vsearch epSeq/table.qza --i-query /home/dimitrios/MetONTIIME/derepSeq/rep-seqs.qza xonomy/taxonomy.--i-reference-reads /home/dimitrios/MetONTIIME/importDb/db_sequences.qza imitrios/MetON--i-reference-taxonomy /home/dimitrios/MetONTIIME/importDb/db_taxonomy.qza --p-perc-identity 0.9 --p-query-cov 0.8 --p-maxaccepts 100 --p-maxrejects 100 --p-maxhits 3 --p-strand 'both' --p-unassignable-label 'Unassigned' --p-threads 6 us: --o-classification /home/dimitrios/MetONTIIME/assignTaxonomy/taxonomy.qza --o-search-results /home/dimitrios/MetONTIIME/assignTaxonomy/search_results.qza 7 else echo "Classifier Vsearch is not supported (choose between Blast and Vsearch)" and fi

    qiime metadata tabulate         --m-input-file /home/dimitrios/MetONTIIME/assignTaxonomy/taxonomy.qza       --o-visualization /home/dimitrios/MetONTIIME/assignTaxonomy/taxonomy.qzv

qiime taxa filter-table 10 Killed --i-table /home/dimitrios/MetONTIIME/derepSeq/table.qza vsearch --i-qu--i-taxonomy /home/dimitrios/MetONTIIME/assignTaxonomy/taxonomy.qza ds /home/d--p-exclude Unassigned tDb/db_seq--o-filtered-table /home/dimitrios/MetONTIIME/derepSeq/table-no-Unassigned.qza

Command exit status: 137

Command output: (empty)

Command error: .command.sh: line 12: 10 Killed qiime feature-classifier classify-consensus-vsearch --i-query /home/dimitrios/MetONTIIME/derepSeq/rep-seqs.qza --i-reference-reads /home/dimitrios/MetONTIIME/importDb/db_sequences.qza --i-reference-taxonomy /home/dimitrios/MetONTIIME/importDb/db_taxonomy.qza --p-perc-identity 0.9 --p-query-cov 0.8 --p-maxaccepts 100 --p-maxrejects 100 --p-maxhits 3 --p-strand 'both' --p-unassignable-label 'Unassigned' --p-threads 6 --o-classification /home/dimitrios/MetONTIIME/assignTaxonomy/taxonomy.qza --o-search-results /home/dimitrios/MetONTIIME/assignTaxonomy/search_results.qza

Work dir: /home/dimitrios/MetONTIIME/work/6b/6c0c8fe0e5f8a5e6d575b338d1f04d

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

MaestSi commented 11 months ago

Hi, sorry for the late reply. First, were you able to run the pipeline with the demo dataset? Second, did you adjust RAM memory and CPUs in the conf file, for either Docker or Singularity (the one you are using) profile? SM

dloukovi commented 11 months ago

Dear Simone, I did not adjust RAM memory and CPUs in the config file (I have already attached the config file in the previous issue...). The error 'killed' comes only when I use the Silva database. When I run the pipeline with NCBI16S (same with Epi2me) database the process is completed, however, it produces only results for taxonomic level 1 (please see the previous issue I have opened). I will also need your help for creating heatmaps, krona graphs and alpha and beta diversity plots. Are you interested in a scientific collaboration? Can you send me your personal e-mail address??

MaestSi commented 11 months ago

In the config file, from lines 175 to 257 you can find the resources allocated for each process. In particular, for each process you can find:

cpus = { params.processName ? <numCPUS> : 1 }
memory = { params.processName ? <GB_RAM>.GB + (2.GB * (task.attempt-1)) : 1.GB }

The first line is used to ask for \<numCPUs> if the process is set to true, or 1 CPU otherwise. The second line is used to ask for \<GB_RAM> GigaBytes of RAM memory and to increase of 2 GB the requested RAM memory in case of process failure, for each attempt, if the process is set to true, or 1 GB otherwise. Therefore, I would advise to increase allocated resources, in particular for the assignTaxonomy process, in case it fails with Silva database. You can send me an e-mail to simone.maestri at unimi.it, to briefly describe the project. I already answered about the results only for taxonomic level 1 in the other open issue. Best, SM

MaestSi commented 11 months ago

Closing the issue due to inactivity.