search

franciscozorrilla / metaGEM

:gem: An easy-to-use workflow for generating context specific genome-scale metabolic models and predicting metabolic interactions within microbial communities directly from metagenomic data
https://franciscozorrilla.github.io/metaGEM/
MIT License
189 stars 41 forks source link

binReassemble gets stop abruptly #100

Closed kunaljaani closed 2 years ago

kunaljaani commented 2 years ago

Hi Francisco,

I am running metaGEM on cloud instance with flag --local. While running binReassemble it gets stop abruptly after processing two out of four samples without producing any error. I am using following command, could you please have a look and suggest some fix?

bash metaGEM.sh -t binReassemble -c 28 -m 240 -h 48 -l

I even tried running the unprocessed samples with bash metaGEM.sh -t binReassemble -c 28 -m 240 -h 48 -l --rerun-incomplet but the processing stops after a while (that to at the same point).

Thank you. Kunal

Screenshot from 2022-03-07 20-25-30

franciscozorrilla commented 2 years ago

Hi Kunal,

That is strange, I wonder if it has to do with reaching your cluster-specific maximum job runtime due to having a large number of bins? You may want to reduce the number per run, how many bins are you trying to reassemble per job/run? I would also look into the reassembly log files to see if any errors are mentioned there, perhaps there is a particular bin that is causing the error?

bash metaGEM.sh -t binReassemble -c 28 -m 240 -h 48 -l --rerun-incomplet

Did you modify the metaGEM.sh script to take in a --rerun-incomplet option? This flag is not part of the metaGEM.sh script.

I hope this helps and let me know if you are still stuck or have made progress!

Best, Francisco

kunaljaani commented 2 years ago

Hi Francisco,

Ya, you are right it was due to the runtime error. I am processing 122 bins for that particular sample and one of the bins (bin.4) was creating some problem but now the job has ended sucessfully. I have a query on similar lines, in such a scenario can we remove the bin from the refinesBins and move ahead? or would you suggest some alternative to omit such bin from the process?

Thank you so much Kunal

franciscozorrilla commented 2 years ago

Good question, in my personal experience I have never had to remove/omit bins, as any poor quality/empty bins will typically be filtered out at some point upstream. However, in such a scenario as you suggest, I would personally remove the troublesome bin and proceed with the remaining MAGs, then go back in the end and see why it failed (check logs, run job manually to observe behavior, etc.) and asses if it is OK to leave out of your analysis.

Hope this helps and good luck!

kunaljaani commented 2 years ago

Thank you very much for the clarification. Fortunately, by fixing the runtime error, I could process the problematic bin during the permissive reassembly.

Thanks a lot for your time. Kunal

franciscozorrilla commented 2 years ago

Great! Closing this issue for now but please re-open if you have further problems