AnantharamanLab / METABOLIC

A scalable high-throughput metabolic and biogeochemical functional trait profiler
179 stars 45 forks source link

Running 7000+MAGs, no progress after Prodigal #174

Open Burgersnow opened 10 months ago

Burgersnow commented 10 months ago

Hello, first of all thank you very much for this very useful tool. I'm having a little trouble reprocessing my data. I had 7000+ MAGs, and after executing commands in the form required by the software, no hmmsearch was performed for 3 days after prodigal ended, and no tmp_run_hmmsearch.sh was generated. It seems the command is stuck somewhere between prodigal and hmmsearch. But when I tested it with 10 MAGs, prodigal immediately followed with hmmsearch. When I used 1600 MAGs tests, it took 5 hours for prodigal to finish running hmmsearch. At this speed, more than 7,000 MAGs could run hmmsearch almost 24 hours after prodigal was completed. I don't know much about perl and wonder if this step between prodigal and hmmsearch takes so long (more than 3 days) when there are a lot of MAGs (7000+MAGs, about 18G in total) to process? Or do you have any other suggestions?

ChaoLab commented 10 months ago

METABOLIC would have problems in processing so many MAGs as we operated it before. Yes, it runs very slow and gets stuck. I will suggest you to run them separately, for example, run 2000 MAGs each time

Burgersnow commented 10 months ago

If I divide these 7000 MAGs into four groups and run the metabolic c.pl program respectively, how can I integrate these four groups of results together to draw metabolic sankey, functional network and other pictures?

adityabandla commented 9 months ago

Running into the same issue with 2000 MAGs

LvbBo commented 1 day ago

If I divide these 7000 MAGs into four groups and run the metabolic c.pl program respectively, how can I integrate these four groups of results together to draw metabolic sankey, functional network and other pictures?

Hi, I encountered the same problem. Have you solved this problem? If so, please tell me the details. Thanks!!!