search

Ecogenomics / BamM

Metagenomics-focused BAM file manipulation
http://ecogenomics.github.io/BamM/
GNU Lesser General Public License v3.0
16 stars 7 forks source link

"bamm parse" exits with Error message, even though "bamm make" works fine #46

Closed jvollme closed 7 years ago

jvollme commented 7 years ago

Hi, I installed BamM 1.7.3. I used the included setup.py.

Usingbamm make to map against a reference worked fine. But when I tried to extract coverage or linkage information using bamm parse (bamm parse -b *.bam -c testcov.tab -m opmean -t 8), i get several error messages, mostly mentioning that "libhts.so.1" was not found (complete error message below, but it is quite long).

Indeed, I can't find this file on my machine, but shouldn't the hts library have come bundled with BamM (at least it says so in the manual)? What am I doing wrong?

The exact Error-message: 

********************************************************************************

ERROR:

Problem importing the BamM c library. This typically means that BamM is not installed correctly.
Please check the installation logs for more details.
If you don't have the installation logs then please try to reinstall BamM and look at the output.
Looking for the c library at: /usr/local/lib/python2.7/dist-packages/bamm/libBamM.a

********************************************************************************

********************************************************************************

ERROR:

Problem importing the BamM c library. This typically means that BamM is not installed correctly.
Please check the installation logs for more details.
If you don't have the installation logs then please try to reinstall BamM and look at the output.
Looking for the c library at: /usr/local/lib/python2.7/dist-packages/bamm/libBamM.a

********************************************************************************

Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 94, in externalParseWrapper
********************************************************************************

ERROR:

Problem importing the BamM c library. This typically means that BamM is not installed correctly.
Please check the installation logs for more details.
If you don't have the installation logs then please try to reinstall BamM and look at the output.
Looking for the c library at: /usr/local/lib/python2.7/dist-packages/bamm/libBamM.a

********************************************************************************

    CW = CWrapper()
  File "/usr/local/lib/python2.7/dist-packages/bamm/cWrapper.py", line 464, in __init__
Process Process-5:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
********************************************************************************

ERROR:

Problem importing the BamM c library. This typically means that BamM is not installed correctly.
Please check the installation logs for more details.
If you don't have the installation logs then please try to reinstall BamM and look at the output.
Looking for the c library at: /usr/local/lib/python2.7/dist-packages/bamm/libBamM.a

********************************************************************************

    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
    self.libPMBam = c.cdll.LoadLibrary(c_lib)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 94, in externalParseWrapper
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    CW = CWrapper()
  File "/usr/local/lib/python2.7/dist-packages/bamm/cWrapper.py", line 464, in __init__
    self.libPMBam = c.cdll.LoadLibrary(c_lib)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
    self._handle = _dlopen(self._name, mode)
OSError: libhts.so.1: cannot open shared object file: No such file or directory
OSError: libhts.so.1: cannot open shared object file: No such file or directory
********************************************************************************

ERROR:

Problem importing the BamM c library. This typically means that BamM is not installed correctly.
Please check the installation logs for more details.
If you don't have the installation logs then please try to reinstall BamM and look at the output.
Looking for the c library at: /usr/local/lib/python2.7/dist-packages/bamm/libBamM.a

********************************************************************************

********************************************************************************

ERROR:

Problem importing the BamM c library. This typically means that BamM is not installed correctly.
Please check the installation logs for more details.
If you don't have the installation logs then please try to reinstall BamM and look at the output.
Looking for the c library at: /usr/local/lib/python2.7/dist-packages/bamm/libBamM.a

********************************************************************************

Process Process-4:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 94, in externalParseWrapper
    CW = CWrapper()
  File "/usr/local/lib/python2.7/dist-packages/bamm/cWrapper.py", line 464, in __init__
Process Process-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.libPMBam = c.cdll.LoadLibrary(c_lib)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
    return self._dlltype(name)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 94, in externalParseWrapper
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libhts.so.1: cannot open shared object file: No such file or directory
    CW = CWrapper()
  File "/usr/local/lib/python2.7/dist-packages/bamm/cWrapper.py", line 464, in __init__
    self.libPMBam = c.cdll.LoadLibrary(c_lib)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libhts.so.1: cannot open shared object file: No such file or directory
Process Process-9:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
Process Process-6:
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
Traceback (most recent call last):
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 94, in externalParseWrapper
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
    CW = CWrapper()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/usr/local/lib/python2.7/dist-packages/bamm/cWrapper.py", line 464, in __init__
********************************************************************************

ERROR:

Problem importing the BamM c library. This typically means that BamM is not installed correctly.
Please check the installation logs for more details.
If you don't have the installation logs then please try to reinstall BamM and look at the output.
Looking for the c library at: /usr/local/lib/python2.7/dist-packages/bamm/libBamM.a

********************************************************************************

    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 94, in externalParseWrapper
    self.libPMBam = c.cdll.LoadLibrary(c_lib)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
********************************************************************************

ERROR:

Problem importing the BamM c library. This typically means that BamM is not installed correctly.
Please check the installation logs for more details.
If you don't have the installation logs then please try to reinstall BamM and look at the output.
Looking for the c library at: /usr/local/lib/python2.7/dist-packages/bamm/libBamM.a

********************************************************************************

    CW = CWrapper()
  File "/usr/local/lib/python2.7/dist-packages/bamm/cWrapper.py", line 464, in __init__
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libhts.so.1: cannot open shared object file: No such file or directory
    self.libPMBam = c.cdll.LoadLibrary(c_lib)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libhts.so.1: cannot open shared object file: No such file or directory
Process Process-8:
Traceback (most recent call last):
Process Process-7:
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 94, in externalParseWrapper
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 94, in externalParseWrapper
    CW = CWrapper()
    CW = CWrapper()
  File "/usr/local/lib/python2.7/dist-packages/bamm/cWrapper.py", line 464, in __init__
  File "/usr/local/lib/python2.7/dist-packages/bamm/cWrapper.py", line 464, in __init__
    self.libPMBam = c.cdll.LoadLibrary(c_lib)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    self.libPMBam = c.cdll.LoadLibrary(c_lib)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    return self._dlltype(name)
    self._handle = _dlopen(self._name, mode)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libhts.so.1: cannot open shared object file: No such file or directory
OSError: libhts.so.1: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/usr/local/bin/bamm", line 660, in <module>
    doWork(args)
  File "/usr/local/bin/bamm", line 222, in doWork
    verbose=args.verbose)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 396, in parseBams
    self.collapseBFIs(BFI_list)
  File "/usr/local/lib/python2.7/dist-packages/bamm/bamParser.py", line 496, in collapseBFIs
    self.BFI = BFI_list[baseBFI_index]
  File "<string>", line 2, in __getitem__
  File "/usr/lib/python2.7/multiprocessing/managers.py", line 774, in _callmethod
    raise convert_to_error(kind, result)
IndexError: list index out of range
jvollme commented 7 years ago

Oh, wait, I now did find "libhts.so.1", but it is in my anaconda path (because I use virtual environments from conda). Maybe I'll have to recompile setting the htslib flag for setup.py? But still i wonder if not the bundled version should have been installed if BamM is using the systems library paths instead of the currently active conda environment's?

For instance, the BamM python libraries were installed in /usr/local/lib/python2.7/dist-packages instead of my currently active environments specific python installation (which should be the only one listed in PYTHON_PATH). Can/should I set the exact preferred python-path for setup.py similar to the htslib paths? However, this does not seem to matter so much, because as i mentioned above bamm make works just fine as it is (only bamm parse seems to be giving me trouble).

wwood commented 7 years ago

Hi there, I'm a little confused about how you installed bamm, but I can say that where parse uses the C part (and hence htslib). make is pure python. So that is probably why you are seeing a difference.

You could try specifying --with-libhts-lib and --with-libhts-inc to setup.py, as you suggest. Good luck.

jvollme commented 7 years ago

Actually, before trying recompiling I just now tried explicitely adding the conda-environments htslib path i found to LD_LIBRARY_PATH, and now it seems to work, so that seems to be fine for me.

Concerning the way I installed BamM was by just cloning the BamM git and then running "setup.py" using sudo. Now I realize that I must have used sudo without the "-E" argument, so it would not have my conda environments PYTHON_PATH, explaining why it was installed in my systems general Python_path. Simple mistake, sorry for posting prematurely...