jtamames / SqueezeMeta

A complete pipeline for metagenomic analysis
GNU General Public License v3.0
348 stars 81 forks source link

Step 1 crashes no matter what we tried #675

Closed paleome closed 1 year ago

paleome commented 1 year ago

Hello again. There seems to be an issue at step 1. If I understand the error correctly, there is no assembly created in the Results subdirectory and the script cannot continue. We tried different samples, different assembly strategies (c-assembly, seqmerge, and now sequential). We keep getting the same error. Please see screen shot below as well as the logfile attached.

Marco

Thanks for your help!

(SqueezeMeta) marco@marco-HP-Z840-Workstation:~/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts$ SqueezeMeta.pl -m sequential -s ~/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/MC_SGM17Apr23/data/Mulu_DC_biofilms2.samples -f ~/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/MC_SGM17Apr23/data/raw_fastq --nopfam --Euk -miniden 50 -t 48 -binners metabat2 -a megahit


SqueezeMeta v1.6.2, March 2023 - (c) J. Tamames, F. Puente-Sánchez CNB-CSIC, Madrid, SPAIN

Please cite: Tamames & Puente-Sanchez, Frontiers in Microbiology 9, 3349 (2019). doi: https://doi.org/10.3389/fmicb.2018.03349

Run started Fri Apr 28 17:05:35 2023 in sequential mode 4 metagenomes found: D3B_Ye D5B_Ac D6B_Gu D3B

--- SAMPLE D3B_Ye --- Now creating directories Reading configuration from /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/SqueezeMeta_conf.pl Running trimmomatic (Bolger et al 2014, Bioinformatics 30(15):2114-20) for quality filtering Parameters: [0 seconds]: STEP1 -> RUNNING ASSEMBLY: 01.run_all_assemblies.pl (megahit) Working for sample D3B_Ye Preparing files for pair1: cat /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/MC_SGM17Apr23/data/raw_fastq/D3B_Ye_R1.fastq.gz > /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/raw_fastq/par1.fastq.gz Preparing files for pair2: cat /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/MC_SGM17Apr23/data/raw_fastq/D3B_Ye_R2.fastq.gz > /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/raw_fastq/par2.fastq.gz Running assembly with megahit: perl /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/lib/SqueezeMeta/assembly_megahit.pl /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye D3B_Ye /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/raw_fastq/par1.fastq.gz /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/raw_fastq/par2.fastq.gz Error running command: /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/bin/megahit/megahit -1 /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/raw_fastq/par1.fastq.gz -2 /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/raw_fastq/par2.fastq.gz -t 48 -o /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/megahit >> /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/syslog 2>&1 at /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/lib/SqueezeMeta/assembly_megahit.pl line 36. Assembly not present in /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/results/01.D3B_Ye.fasta, exiting Stopping in STEP1 -> 01.run_all_assemblies.pl. Program finished abnormally

If you don't know what went wrong or want further advice, please look for similar issues in https://github.com/jtamames/SqueezeMeta/issues Feel free to open a new issue if you don't find the answer there. Please add a brief description of the problem and upload the /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/syslog file (zip it first) Died at /home/marco/miniconda3/envs/SqueezeMeta/bin/SqueezeMeta.pl line 941. syslog.zip

jtamames commented 1 year ago

Hi Could you paste the content of the data/megahit/log file? Best, J

paleome commented 1 year ago

2023-04-28 17:06:44 - MEGAHIT v1.2.9 2023-04-28 17:06:44 - Using megahit_core with POPCNT and BMI2 support 2023-04-28 17:06:44 - Convert reads to binary library 2023-04-28 17:06:44 - command /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/bin/megahit/megahit_core buildlib /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/megahit/tmp/reads.lib /home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/megahit/tmp/reads.lib 2023-04-28 17:09:27 - b'INFO sequence/io/sequence_lib.cpp : 77 - Lib 0 (/home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/raw_fastq/par1.fastq.gz,/home/marco/miniconda3/envs/SqueezeMeta/SqueezeMeta/scripts/D3B_Ye/data/raw_fastq/par2.fastq.gz): pe, 123960326 reads, 528 max length' 2023-04-28 17:09:27 - b'INFO utils/utils.h : 152 - Real: 162.5078\tuser: 114.0153\tsys: 18.3320\tmaxrss: 245656' 2023-04-28 17:09:27 - Error occurs when reading inputs

jtamames commented 1 year ago

Hello I myself am having issues with megahit lately. Still trying to figure out what's wrong. Meanwhile, could you try spades instead? Use the -a spades option when running SqueezeMeta Best, J

paleome commented 1 year ago

Hi again. We tried spades but it is extremely slow. We decided to quit the run after it was stuck on Step 1 for 10 days. This is why we tried megahit instead.

fpusan commented 1 year ago

Hi, Just in case make sure that none of your fastq files are corrupted.

fpusan commented 1 year ago

Is this happening also with the test files we provide when downloading the databases?

paleome commented 1 year ago

My apologies for the delayed response. Yes it was also the case with those test files. You mentioned that you were also having issues with megahit lately. Was this related to my issue and/or have you found the issue?

jtamames commented 1 year ago

Hum, then it is system-related, since we tried megahit using these files and it works fine. Could it be a memory-related problem?

paleome commented 1 year ago

Hi James,

We have 512 Gb RAM installed. I will try it again with those samples. I am actually not 100% sure if we tried megahit on these test samples before

On 11 May 2023, at 8:39 pm, Javier Tamames @.***> wrote:

Hum, then it is system-related, since we tried megahit using these files and it works fine. Could it be a memory-related problem?

— Reply to this email directly, view it on GitHubhttps://github.com/jtamames/SqueezeMeta/issues/675#issuecomment-1543924224, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHOISX7XPLQTPAMEOJPZOD3XFTMXVANCNFSM6AAAAAAXO7MRII. You are receiving this because you authored the thread.Message ID: @.***>

fpusan commented 1 year ago

Closing due to lack of activity, feel free to reopen!