jtamames / SqueezeMeta

A complete pipeline for metagenomic analysis
GNU General Public License v3.0
346 stars 81 forks source link

Stopping in STEP4 —> 04.rundiamond .pl Program finished abnormally #762

Closed da-gunjan-arya closed 6 months ago

da-gunjan-arya commented 6 months ago

All checks successfully after running test_install.pl When I tried to running with other sample in sequential mode for flye assembly it Stopping in STEP4 —> 04.rundiamond .pl Program finished abnormally at line 941 image

reading your previous issues #707 I let it auto detect the -b according to my RAM viz. (We will set Diamond block size to 15.2 [Gb RAM/8, Max 16] ). It was still producing the same error. This is the error "Unexpected end of input" detected in diamond.nr.log

image

and i'm attaching the following syslog file

syslog.zip

fpusan commented 6 months ago

This doesn't seem to be a problem with memory. Rather it would seem the problem happens when DIAMOND is trying to load the reference database. Did you get the database using download_databases.pl?

fpusan commented 6 months ago

Also, did the pipeline work with the test datasets?

da-gunjan-arya commented 6 months ago

Database was re-downloaded using download_databases.pl and yes it works for test dataset. The script is able to load reference db and perform diamond, but now it stops at step 10. viz. bowtie2-align (in this it consumes up all the RAM)

image
fpusan commented 6 months ago

Please help me confirm that I understood your issue correctly. So after re-downloading the database: 1) SqueezeMeta works for the test dataset and the pipeline completes fully. 2) With your own dataset, step 4 now also works correctly. 3) However, with your own dataset, the pipeline now stops at step 10 in which bowtie2 is consuming all the RAM. Are these three points correct?

da-gunjan-arya commented 6 months ago

Yes, all points are correct.

fpusan commented 6 months ago

Can you manually run the command producing the error (shown after the "An error occurred during mapping with command:" message) removing the --quiet flag? Let's see if we get a more specific error message

fpusan commented 6 months ago

How much RAM do you have available?

da-gunjan-arya commented 6 months ago

I will try running that. Available RAM 124gb

fpusan commented 6 months ago

I see that you used flye as an assembler. Are these long reads? If so you should be using minimap2 for mapping. Check the ReadMe for instructions on how to do this.

da-gunjan-arya commented 6 months ago

I've tried running it with -map minimap2-ont and it was sorted but since sequencing data had more short reads even though its nanopore, I preferred using default parameters.

jtamames commented 6 months ago

Do you mean that minimap2-ont is working for you? It does not matter the length of the reads, minion reads have particular errors that makes it very advisable the usage of minimap2-ont

da-gunjan-arya commented 6 months ago

Yes it's working for now with all the samples I tried with. The pipepline runs without any errors. Thanks for the help.

fpusan commented 6 months ago

Glad to hear! closing issue