Bin_refinement module error. Process killed at finalizing the refined bins.

[gkong1]

Hi,

I ran the bin refinement module and everything went smoothly until it was finalizing the refined bins. I obtained the following error: Uncaught exception: Sys_error("binsO.checkm/storage/tree/concatenated.pplacer.json: No such file or directory") Fatal error: exception Sys_error("binsO.checkm/storage/tree/concatenated.pplacer.json: No such file or directory")

Below are the details during the last part of the bin refinement module.

------------------------------------------------------------------------------------------------------------------------
-----                     You will find the best non-reassembled versions of the bins in binsO                     -----
------------------------------------------------------------------------------------------------------------------------

########################################################################################################################
#####                                          FINALIZING THE REFINED BINS                                         #####
########################################################################################################################

------------------------------------------------------------------------------------------------------------------------
-----                                        Re-running CheckM on binsO bins                                       -----
------------------------------------------------------------------------------------------------------------------------

*******************************************************************************
 [CheckM - tree] Placing bins in reference genome tree.
*******************************************************************************

  Identifying marker genes in 6 bins with 11 threads:
    Finished processing 6 of 6 (100.00%) bins.
  Saving HMM info to file.

  Calculating genome statistics for 6 bins with 11 threads:
    Finished processing 6 of 6 (100.00%) bins.

  Extracting marker genes to align.
  Parsing HMM hits to marker genes:
    Finished parsing hits for 6 of 6 (100.00%) bins.
  Extracting 43 HMMs with 11 threads:
    Finished extracting 43 of 43 (100.00%) HMMs.
  Aligning 43 marker genes with 11 threads:
    Finished aligning 43 of 43 (100.00%) marker genes.

  Reading marker alignment files.
  Concatenating alignments.
  Placing 6 bins into the genome tree with pplacer (be patient).
Killed
Uncaught exception: Sys_error("binsO.checkm/storage/tree/concatenated.pplacer.json: No such file or directory")
Fatal error: exception Sys_error("binsO.checkm/storage/tree/concatenated.pplacer.json: No such file or directory")

  { Current stage: 0:06:31.217 || Total: 0:06:31.217 }

*******************************************************************************
 [CheckM - lineage_set] Inferring lineage-specific marker sets.
*******************************************************************************

  Reading HMM info from file.
  Parsing HMM hits to marker genes:
    Finished parsing hits for 6 of 6 (100.00%) bins.

  Determining marker sets for each genome bin.

Unexpected error: <type 'exceptions.IOError'>
Traceback (most recent call last):
  File "/mnt/miniconda3/envs/metawrap-env/bin/checkm", line 712, in <module>
    checkmParser.parseOptions(args)
  File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/checkm/main.py", line 1245, in parseOptions
    self.lineageSet(options)
  File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/checkm/main.py", line 230, in lineageSet
    resultsParser, options.unique, options.multi)
  File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/checkm/treeParser.py", line 477, in getBinMarkerSets
    tree = dendropy.Tree.get_from_path(treeFile, schema='newick', rooting="force-rooted", preserve_underscores=True)
  File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/dendropy/datamodel/basemodel.py", line 216, in get_from_path
    with open(src, "r", newline=None) as fsrc:
  File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/dendropy/utility/filesys.py", line 61, in pre_py34_open
    buffering=buffering)
IOError: [Errno 2] No such file or directory: 'binsO.checkm/storage/tree/concatenated.tre'

************************************************************************************************************************
*****                             Something went wrong with running CheckM. Exiting...                             *****
************************************************************************************************************************

Thanks in advance!

[ursky]

Hey there! It appears like pplacer crashed, likely from running out of memory. How much RAM did you give the metaWRAP run? In my experience, the pplacer run can use upwards of 40GB.

[gkong1]

Hi, I increased the RAM to 64gb and it finished smoothly. Thanks for your help!

[ursky]

Fantastic! Ill add a recommended minimum to the module for the future.