jtamames / SqueezeMeta

A complete pipeline for metagenomic analysis
GNU General Public License v3.0
346 stars 81 forks source link

Step 14 Not Progressing #826

Closed giovrosin closed 1 month ago

giovrosin commented 2 months ago

Hello,

Im having trouble on step 14. The program has run on this step for 11 days straight. I tried restarting the run with more threads but it doesn't seem like its progressing. I checked the /temp/ and the /intermediate/binners/concoct/ files and there seem to be no updates.

Im working with 77 metagenomes and 102 gb total.

I will anex the syslog and the std output.

Any help is appreciated!

log.zip

jtamames commented 2 months ago

Hello Yes, I myself have experienced some trouble with concoct when analyzing big projects as yours. Are you interested in having bins? If not, you could just stop the process and create the contig table and the final stats. If yes, I could tell you a different way of having bins for your samples. Best, J

giovrosin commented 2 months ago

Yes, my project focuses on taxonomic analyses and assignment of bins so this step is important

fpusan commented 2 months ago

Is CONCOCT still running, and if so is it using all the CPU threads at 100%?

giovrosin commented 2 months ago

yes, they're running at 99%

fpusan commented 2 months ago

Then it may be just that the dataset is too complex for CONCOCT. You can either wait, or avoid CONCOCT and use only METABAT2 for getting your bins, since it can be much faster.

jtamames commented 2 months ago

A different approach that you can follow is analyzing the samples in sequential mode instead of coassembly. That will provide individual MAGs for each sample, that you can later merge together (dereplicate) using their ANIs. A program such as dRep can be used for that purpose. In this way you usually get many more bins, because you are able to retrieve these that are abundant in just a few samples, which could be missed using the coassembly

giovrosin commented 2 months ago

Hello!

Thank you for the reply. First I tried restarting the run using metabat2, but as I understand it's running CONCOT still.

Here is the command and syslog:

SqueezeMeta.pl -p SQM_Superior -binners metabat2 --restart -step 13 --force_overwrite -s /home/cstaats/giovana/sqm_illumina/lista_sqm_sup -f /home/cstaats/giovana/sqm_illumina/TratoSuperior -t 50

syslog.zip

fpusan commented 2 months ago

The way to change the configuration mid run is to edit the SqueezeMeta_conf.pl file inside of the projects directory (so SQM_Superior/SqueezeMeta_conf.pl. There, find the variable $binners and remove concoct. Then try to restart again.

fpusan commented 1 month ago

Closing due to lack of activity, feel free to reopen