search

brentp / cyvcf2

cython + htslib == fast VCF and BCF processing
MIT License
382 stars 72 forks source link

undefined symbol: SHA256 during test #123

Open rajkump opened 5 years ago

rajkump commented 5 years ago

Hi, The installation of cyvcf2 is successful. I followed the steps below for installation and everything was success. When I did a test of install I get the undefined symbol: SHA256

cyvcf2$ sudo python2.7 setup.py test /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type' warnings.warn(msg) running test /usr/lib/python2.7/dist-packages/setuptools/dist.py:313: UserWarning: Unbuilt egg for Unknown [unknown version] (/usr/local/lib/python2.7/dist-packages) replace_conflicting=True, /usr/lib/python2.7/dist-packages/setuptools/package_index.py:272: UserWarning: Unbuilt egg for Unknown [unknown version] (/usr/local/lib/python2.7/dist-packages) Environment.init(self,*args,**kw) /usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py:344: UserWarning: Unbuilt egg for Unknown [unknown version] (/usr/local/lib/python2.7/dist-packages) self.local_index = Environment(self.shadow_path + sys.path) Searching for nose Best match: nose 1.3.7 Processing nose-1.3.7-py2.7.egg

Using cyvcf2/.eggs/nose-1.3.7-py2.7.egg running egg_info writing requirements to cyvcf2.egg-info/requires.txt writing cyvcf2.egg-info/PKG-INFO writing top-level names to cyvcf2.egg-info/top_level.txt writing dependency_links to cyvcf2.egg-info/dependency_links.txt writing entry points to cyvcf2.egg-info/entry_points.txt reading manifest file 'cyvcf2.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'cyvcf2.egg-info/SOURCES.txt' running build_ext skipping 'cyvcf2/cyvcf2.c' Cython extension (up-to-date) cyvcf2/.eggs/nose-1.3.7-py2.7.egg/nose/config.py:264: RuntimeWarning: Option 'with-coverage' in config file 'setup.cfg' ignored: excluded by runtime environment warn(msg, RuntimeWarning) Failure: ImportError (cyvcf2/cyvcf2/cyvcf2.so: undefined symbol: SHA256) ... ERROR

====================================================================== ERROR: Failure: ImportError (cyvcf2/cyvcf2/cyvcf2.so: undefined symbol: SHA256)

Traceback (most recent call last): File "/home/praveen/Software/cyvcf2/.eggs/nose-1.3.7-py2.7.egg/nose/loader.py", line 418, in loadTestsFromName addr.filename, addr.module) File "/cyvcf2/.eggs/nose-1.3.7-py2.7.egg/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "cyvcf2/.eggs/nose-1.3.7-py2.7.egg/nose/importer.py", line 94, in importFromDir mod = load_module(partfqname, fh, filename, desc) File "cyvcf2/cyvcf2/init.py", line 1, in from .cyvcf2 import (VCF, Variant, Writer, r as r_unphased, par_relatedness, ImportError:cyvcf2/cyvcf2/cyvcf2.so: undefined symbol: SHA256

brentp commented 5 years ago

you'll need to install libcrypto and libssl.

rajkump commented 5 years ago

Thanks for the help. I use Ubuntu 16 LTS. I have "libcrypto++-dev is already the newest version" and "libssl-dev is already the newest version (1.0.2g-1ubuntu4.15)."

rajkump commented 5 years ago

I found both of these libs are in "/usr/share/doc/". I added these to "/etc/ld.so.conf" like include /usr/share/doc/ and ran ldconfig. Still during test I get undefined symbol: SHA256) ... ERROR.

brentp commented 5 years ago

they are probably not in /usr/share/doc, find where the .so files with, e.g.: locate libcrypto.so

rajkump commented 5 years ago

Hi, Thanks for the help. I did locate "/usr/local/lib/libssl.so" and "/usr/local/lib/libcrypto.so". Indeed, this path was already included in "/etc/ld.so.conf".

It was included like include /usr/local/lib/

I still get the error during test "ImportError: ****/cyvcf2/cyvcf2/cyvcf2.so: undefined symbol: SHA256".

Also, I would like to let you know that there was another Import error initially like "ImportError: **/cyvcf2/cyvcf2/cyvcf2.so: undefined symbol: EVP_sha1". This I gathered from other posts that it may be because of old 'htslib' packaged with cyvcf2. So I did the following steps to get rid of the "EVP ERROR". Then I ended up with "SHA256 ERROR".

The steps I followed to get current version of htslib mv htslib htslib-bkup git clone --recursive https://github.com/samtools/htslib cd htslib autoheader autoconf ./configure --enable-libcurl make cd .. sudo -H pip2.7 install -e .

was successfully installed Then during test cyvcf2$ sudo python2.7 setup.py test /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type' warnings.warn(msg) running test /usr/lib/python2.7/dist-packages/setuptools/dist.py:313: UserWarning: Unbuilt egg for Unknown [unknown version] (/usr/local/lib/python2.7/dist-packages) replace_conflicting=True, /usr/lib/python2.7/dist-packages/setuptools/package_index.py:272: UserWarning: Unbuilt egg for Unknown [unknown version] (/usr/local/lib/python2.7/dist-packages) Environment.init(self,*args,**kw) /usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py:344: UserWarning: Unbuilt egg for Unknown [unknown version] (/usr/local/lib/python2.7/dist-packages) self.local_index = Environment(self.shadow_path + sys.path) Searching for nose Best match: nose 1.3.7 Processing nose-1.3.7-py2.7.egg

Using cyvcf2/.eggs/nose-1.3.7-py2.7.egg running egg_info writing requirements to cyvcf2.egg-info/requires.txt writing cyvcf2.egg-info/PKG-INFO writing top-level names to cyvcf2.egg-info/top_level.txt writing dependency_links to cyvcf2.egg-info/dependency_links.txt writing entry points to cyvcf2.egg-info/entry_points.txt reading manifest file 'cyvcf2.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'cyvcf2.egg-info/SOURCES.txt' running build_ext skipping 'cyvcf2/cyvcf2.c' Cython extension (up-to-date) cyvcf2/.eggs/nose-1.3.7-py2.7.egg/nose/config.py:264: RuntimeWarning: Option 'with-coverage' in config file 'setup.cfg' ignored: excluded by runtime environment warn(msg, RuntimeWarning) Failure: ImportError (cyvcf2/cyvcf2/cyvcf2.so: undefined symbol: SHA256) ... ERROR

====================================================================== ERROR: Failure: ImportError (cyvcf2/cyvcf2/cyvcf2.so: undefined symbol: SHA256)

Traceback (most recent call last): File "cyvcf2/.eggs/nose-1.3.7-py2.7.egg/nose/loader.py", line 418, in loadTestsFromName addr.filename, addr.module) File "cyvcf2/.eggs/nose-1.3.7-py2.7.egg/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "cyvcf2/.eggs/nose-1.3.7-py2.7.egg/nose/importer.py", line 94, in importFromDir mod = load_module(partfqname, fh, filename, desc) File "***/cyvcf2/cyvcf2/init.py", line 1, in from .cyvcf2 import (VCF, Variant, Writer, r as r_unphased, par_relatedness, ImportError: ***/cyvcf2/cyvcf2/cyvcf2.so: undefined symbol: SHA256

Ran 1 test in 0.006s

FAILED (errors=1) ***/cyvcf2$

brentp commented 5 years ago

do you have a libssl.so available?

brentp commented 5 years ago

i would recommend to install using bioconda. that will save you from this trouble

eroller commented 4 years ago

I'm hitting this same error on Ubuntu 18.04 (WSL) with python3.8. I've installed cyvcf2 0.20.9 using pip.

I have a successful install of cyvcf2 on centos7 and running ldd on the cyvcf2.so file shows it depends on libcrypt.so and libssl.so, However, on the Ubuntu build of cyvcf2.so ldd does not show libcrypto.so or libssl.so, although there is libcrypt.so and libhcrypto.so. I'm not sure if that matters, It seems like maybe the specific version of htslib requires a different libcrpto.so or libssl.so than what I have on my system? I'm not sure why cyvcf2 build doesn't give a link error but also doesn't show any missing library dependencies from ldd.

ccwang002 commented 4 years ago

Here is how I install cyvcf2 inside an Ubuntu 18.04 docker image with python3.8. The build passes all the tests. 

docker run -it --rm ubuntu:18.04 bash -l

apt update
apt install build-essential zlib1g-dev libbz2-dev liblzma-dev libssl-dev libcurl4-openssl-dev autoconf
apt install python3.8-venv python3.8-dev python3-pip

git clone --recursive https://github.com/brentp/cyvcf2
cd htslib
autoheader
autoconf
./configure --enable-libcurl --enable-s3 --enable-lzma --enable-bz2
make -j4

cd ..
python3.8 -m pip install --upgrade pip
python3.8 -m pip install -r requirements.txt
CYTHONIZE=1 python3.8 setup.py test
# ...
# ----------------------------------------------------------------------
# Ran 90 tests in 1.052s
# 
# OK

Running the tests currently requires building cyvcf2 from source.

eroller commented 4 years ago

Thanks. I confirmed that works. I also installed cyvcf2 to my site python3.8 site packages using CYTHONIZE=1 pip install -e . as mentioned in the README so that cyvcf2 could be used by dependent modules.

I would recommend keeping this issue open since the pip install cyvcf2 method mentioned in the README doesn't seem to work for all systems (e.g. Ubuntu 18.04). I had installed htslib 1.9 before running pip install cyvcf2 and it resulted in the same undefined symbol: SHA256 error.

cvravi9 commented 3 years ago

For anyone getting this error when installing vcfstats, please use the combination of the above two comments to get cyvcf2 installed in-order to use the vcfstats:

apt update apt install build-essential zlib1g-dev libbz2-dev liblzma-dev libssl-dev libcurl4-openssl-dev autoconf apt install python3.8-venv python3.8-dev python3-pip

git clone --recursive https://github.com/brentp/cyvcf2 cd cyvcf2/htslib autoheader autoconf ./configure --enable-libcurl --enable-s3 --enable-lzma --enable-bz2 make -j4

cd .. python3.8 -m pip install --upgrade pip python3.8 -m pip install -r requirements.txt CYTHONIZE=1 pip install -e .

Thank you guys!

CholoTook commented 2 years ago

Can you make a PR on the install document please?