list of library depencies for reassembly module on HPC cluster

Hello Gherman,

I am running all steps from the metaWRAP tool on a HPC cluster. That means that not only all resources (time, memory, cores) should be requested before the run (based on prediction) but also all of the dependencies.

So far, every dependency was satisfied by the installation and simply loading the conda metawrap_env was enough to get everything running, except for the reassembly module. Apparently there libraries that are not available and still they are being required.

Example: When running the reassembly module for a given set of samples. I got the following line on my *err output file:

$ more seawater_reassembly_bins*.err | grep Import
ImportError: libXext.so.6: cannot open shared object file: No such file or directory

So what I did was to " manually" load this library for the run with "module load libxext" and run again. Then I got a second error:

$ more seawater_reassembly_bins*.err | grep Import
ImportError: libSM.so.6: cannot open shared object file: No such file or directory

I assume, then, there are a few libraries that are not available.

I will append more details from the last run .err file:

$ tail -n 40 sponge_reassembly_bins-4808018.err

  Calculating AAI between multi-copy marker genes.

  Reading HMM info from file.
  Parsing HMM hits to marker genes:
    Finished parsing hits for 14 of 14 (100.00%) bins.

  { Current stage: 0:00:08.592 || Total: 0:12:47.236 }

*******************************************************************************
 [CheckM - bin_qa_plot] Creating bar plot of bin quality.
*******************************************************************************

  Calculating AAI between multi-copy marker genes.
  Plotting bin completeness, contamination, and strain heterogeneity.

  Plot written to: /data/msb/LEMAM/rita_polonia/BIN_REASSEMBLY/sponge/reassembled_bins.plot/bin_qa_plot.png

  { Current stage: 0:03:05.834 || Total: 0:03:05.834 }
Traceback (most recent call last):
  File "/data/msb/tools/metawrap/metawrap_env/bin/metawrap-scripts/plot_reassembly.py", line 6, in <module>
    import matplotlib.pyplot as plt
  File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/pyplot.py", line 114, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
  File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 32, in pylab_setup
    globals(),locals(),[backend_name],0)
  File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/backend_qt4agg.py", line 18, in <module>
    from .backend_qt5agg import FigureCanvasQTAggBase as _FigureCanvasQTAggBase
  File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/backend_qt5agg.py", line 15, in <module>
    from .backend_qt5 import QtCore
  File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/backend_qt5.py", line 31, in <module>
    from .qt_compat import QtCore, QtGui, QtWidgets, _getSaveFileName, __version__
  File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/qt_compat.py", line 124, in <module>
    from PyQt4 import QtCore, QtGui
ImportError: libSM.so.6: cannot open shared object file: No such file or directory

real    601m30.487s
user    1709m57.309s
sys 81m37.775s

And for the same run, the .out file

$ tail -n 30 sponge_reassembly_bins-4808018.out
------------------------------------------------------------------------------------------------------------------------
-----                                          Finalizing CheckM stats...                                          -----
------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------
-----           Making CheckM plot of /data/msb/LEMAM/rita_polonia/BIN_REASSEMBLY/sponge/reassembled_bins          -----
-----                                                     bins                                                     -----
------------------------------------------------------------------------------------------------------------------------

    Plotting bin 14 of 14 (100.00%) bins.

------------------------------------------------------------------------------------------------------------------------
-----                            you will find the info on the final reassembled bins in                           -----
-----           /data/msb/LEMAM/rita_polonia/BIN_REASSEMBLY/sponge/reassembled_bins.stats, and a figure            -----
-----          summarizing it in /data/msb/LEMAM/rita_polonia/BIN_REASSEMBLY/sponge/reassembled_bins.png           -----
------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------
-----                      making reassembly N50, compleiton, and contamination summary plots.                     -----
------------------------------------------------------------------------------------------------------------------------

************************************************************************************************************************
*****                  Something went wrong with plotting the reassembly summary plots. Exiting...                 *****
************************************************************************************************************************

end reassembly bins
Wed Sep 26 02:38:54 CEST 2018

Again, I assume this is the problem. Could I maybe get a list of the dependencies that are required? I tried to find the list of the libraries needed but it was a little challenging for me.

Thanks again for support!!!

Best, Rodolfo

bxlab / metaWRAP

list of library depencies for reassembly module on HPC cluster #46