Step 4 Error - Running Diamond

jtamames / SqueezeMeta

A complete pipeline for metagenomic analysis

GNU General Public License v3.0

373 stars 80 forks source link

Step 4 Error - Running Diamond #672

Closed Ravi-WSU closed 1 year ago

Ravi-WSU commented 1 year ago

Hello, I'm getting an error message in step 4, I'm not sure what to do so I was hoping you could help. This analysis was running on the GRID utilizing 1 node, 15 cpus, and 700 GB memory. I've attached the image of the output file below and the syslog is also attached.

syslog.txt

fpusan commented 1 year ago

Can you share the /nfs/prb/Gut_metagenomics_7202/Renamed/SQM_PE_Subset/temp/diamond.nr.log file with us?

Ravi-WSU commented 1 year ago

Hello Fernando, the file is attached.

diamond.nr.log

fpusan commented 1 year ago

Ok, so you have the following error

Computing alignments... terminate called after throwing an instance of 'std::runtime_error'
  what():  Format error: Invalid varint encoding.

Apparently this has been fixed in the latest version of DIAMOND as per https://github.com/bbuchfink/diamond/issues/653

So you would need to do the following:

1) Download the latest DIAMOND release (linux 64) from their github repo. 2) Uncompress and get the new diamond executable. Copy it to /wsu/home/fw/fw43/fw4347/.conda/envs/SqueezeMeta/SqueezeMeta/bin/diamond in order to replace the old executable that we redistribute with SqueezeMeta. 3) Restart the pipeline with the --restart flag (it should automatically re-attempt from the failed step).

Ravi-WSU commented 1 year ago

That seemed to work, the run has restarted properly. Thanks for your help, Fernando.

fpusan commented 1 year ago

Ok, but let's wait until it finishes before celebrating. Please let me know when/if finishes correctly. If it works, then I will add the new DIAMOND version to SqueezeMeta.

Ravi-WSU commented 1 year ago

I guess I got a little too excited! However, it failed again at the same step. The output image is below and the diamond log is also attached.

diamond.nr.log

fpusan commented 1 year ago

Ok, then try editing the $blocksize variable in /nfs/prb/Gut_metagenomics_7202/Renamed/SQM_PE_Subset/SqueezeMeta_conf.pl and setting it to 1, then restarting.

Ravi-WSU commented 1 year ago

I changed the $blocksize variable and restarted the run, will let you know if anything else comes up.

Ravi-WSU commented 1 year ago

Ok, but let's wait until it finishes before celebrating. Please let me know when/if finishes correctly. If it works, then I will add the new DIAMOND version to SqueezeMeta.

Hello, this run has finally been completed and the update to the new version of Diamond was successful. Thank you for all your help Fernando.

fpusan commented 1 year ago

It took two weeks to complete? wow. To be fair the blocksize 1 makes thing much slower

Ravi-WSU commented 1 year ago

It had stopped on me a few times in the process so I had to restart, so it wasn't two weeks entirely. It actually made it past Diamond fairly quick, but some other steps took time.

fpusan commented 1 year ago

Ah ok! Then I don't think the new DIAMOND version made the trick. Rather, in this case what helped was lowering the blocksize. Anyways, glad to hear it worked eventually!