search

jtamames / SqueezeMeta

A complete pipeline for metagenomic analysis
GNU General Public License v3.0
346 stars 81 forks source link

Stopping in STEP4 -> 04.rundiamond.pl. Program finished abnormally #782

Closed diegorojasa closed 4 months ago

diegorojasa commented 5 months ago

Hola!

I'm trying to analize two metagenomes (4 files, 13.5GB) in my personal computer (16GB of RAM). I've installed everything on an external disk (931GB total, 330GB of free space after running download_databases.pl) and 'All checks successful' after running test_install.pl. Step1, 2 and 3 run without problem, but Step4 load about 10 minutes and after that it says Program finished abnormally ...... Died at /home/diegorojasadones2/anaconda3/envs/SqueezeMeta/bin/SqueezeMeta.pl line 941.

I've tried to dive in into but i can not find a way to solve it :'(

Can you help me please? The syslog it's attached!

Thanks in advance!

Diego Rojas A

syslog.zip

diegorojasa commented 5 months ago

ooh, i forgot to tell you, i installed everything today and the error it's from today too, i don't know if that helps :(

fpusan commented 5 months ago

Probably a memory issue, check the content of /mnt/d/genomas/resultados_2/temp/diamond.nr.log to see if there is a more specific error message. Also are you running this from Windows/WSL2? If so read this issue about how storing data inside your windows partition (in this case /mnt/d) can cause performance issues in SqueezeMeta.

fpusan commented 4 months ago

I see you just closed it. Did it work? If so what fixed It?

diegorojasa commented 4 months ago

Hi!

I didn't do anything different, i tried multiple times and one of those passed step 4 but stayed for at least 1 week on step 6. For what i could comprehend it stayed in that step because the diamond proccess created 860.000 files located in the intermediate folder and the step 6 itself was using -2 threads (?) so i decided to abort mission, but it's okay because one friends was able to get a virtual environment with better RAM and free disk space (where my troubles now are on step 10, i uploaded a new issue about it)

Thanks!

fpusan commented 4 months ago

The very slow speed / CPU usage in step 6 was because it was IO-bound, and this was related to you having the databases in the Windows partition of a WSL2 install, as I mentioned in my previous comment

See https://github.com/jtamames/SqueezeMeta/issues/695 for further discussion on this