wiggletools.c -lcrypto needed for compilation.

[marcovth]

git clone https://github.com/dahlo/WiggleTools

cd WiggleTools 
# make ... gives error below ...  -lcrypto is missing from the cc command 
# ~/biotools/WiggleTools/src$ cc -g -Wall -O3 -std=gnu99 -L../lib -L../../libBigWig -L../../htslib wiggletools.c -lwiggletools -l:libBigWig.a -lcurl -l:libhts.a -lgsl  -lgslcblas -lz -lpthread -lm -llzma -lbz2 -o ../bin/wiggletools
# /usr/bin/ld: ../../htslib/libhts.a(hfile_s3.o): undefined reference to symbol 'EVP_sha256@@OPENSSL_3.0.0'
# /usr/bin/ld: /lib/x86_64-linux-gnu/libcrypto.so.3: error adding symbols: DSO missing from command line
# collect2: error: ld returned 1 exit status

# Do this instead ...
cd src
cc -g -Wall -O3 -std=gnu99 -L../lib -L../../libBigWig -L../../htslib wiggletools.c -lwiggletools -l:libBigWig.a -lcurl -l:libhts.a -lgsl  -lgslcblas -lz -lpthread -lm -llzma -lbz2 -o ../bin/wiggletools -lcrypto
cd ..

[sgiorgetti]

Hi @marcovth

Thanks for flagging this up.

I tried to reproduce the error using the original Ensembl/WiggleTools code in a few Linux environments (Ubuntu 22.04, Debian 11, Rocky 8), but everything worked fine instead. Apparently, the crypto library (and headers) is installed by default in these distributions or it comes with the development/building tools.

Did you use a different distro or OS?

[marcovth]

I tried to translate this docker recipe into a singularity script ...

https://github.com/ComparativeGenomicsToolkit/Comparative-Annotation-Toolkit/blob/master/Dockerfile

But in the end, even with my adaptation, it crashed on WiggleTools

====

_BootStrap: docker From: debian:latest IncludeCmd: yes

sudo singularity build --notest ./CAT.singularity.sif ./CAT.singularity.def.txt

%post

~~~~~~~~~~~~~~~~~

this will install all necessary packages and prepare the container

apt-get -y update apt-get -y upgrade

apt-get install -y build-essential libssl-dev libncurses5-dev libcurl4-openssl-dev liblzma-dev libbz2-dev \ libboost-all-dev sqlite3 libsqlite3-0 libsqlite3-dev libgsl0-dev lp-solve liblpsolve55-dev libbamtools-dev wget git unzip

apt-get install -y libbz2-dev liblzma-dev libcurl4-openssl-dev pkg-config libncurses5-dev apt-get -y install libboost-dev

mkdir /root/tools cd /root/tools

htslib

wget https://github.com/samtools/htslib/releases/download/1.16/htslib-1.16.tar.bz2 tar -xvjf htslib-1.16.tar.bz2 mv htslib-1.16 htslib cd htslib ./configure make make install cd .. rm ./htslib-1.16.tar.bz2

bcftools

wget https://github.com/samtools/bcftools/releases/download/1.16/bcftools-1.16.tar.bz2 tar -xvjf bcftools-1.16.tar.bz2 mv bcftools-1.16 bcftools cd bcftools ./configure make make install cd .. rm ./bcftools-1.16.tar.bz2

samtools

wget https://github.com/samtools/samtools/releases/download/1.16/samtools-1.16.tar.bz2 tar -xvjf samtools-1.16.tar.bz2 mv samtools-1.16 samtools cd samtools ./configure make make install cd .. rm ./samtools-1.16.tar.bz2

Augustus

apt-get update apt-get install -y build-essential wget git autoconf

Install dependencies for AUGUSTUS comparative gene prediction mode (CGP)

apt-get install -y libgsl-dev libboost-all-dev libsuitesparse-dev liblpsolve55-dev apt-get install -y libsqlite3-dev libmysql++-dev

Install dependencies for the optional support of gzip compressed input files

apt-get install -y libboost-iostreams-dev zlib1g-dev

Install dependencies for bam2hints and filterBam

apt-get install -y libbamtools-dev zlib1g-dev

Install additional dependencies for bam2wig

apt-get install -y samtools libhts-dev

Install additional dependencies for homGeneMapping and utrrnaseq

apt-get install -y libboost-all-dev

Install additional dependencies for scripts

apt-get install -y cdbfasta diamond-aligner libfile-which-perl libparallel-forkmanager-perl libyaml-perl libdbd-mysql-perl apt-get install -y --no-install-recommends python3-biopython

git clone https://github.com/Gaius-Augustus/Augustus augustus cd augustus && git reset --hard ${AUGUSTUS_COMMIT} echo 'COMPGENEPRED = true' >> ./common.mk echo 'SQLITE = true' >> ./common.mk cd ./auxprogs/homGeneMapping/src && sed 's/# BOOST = true/BOOST = true/g' -i Makefile && sed 's/# SQLITE = true/SQLITE = true/g' -i Makefile cd ../../../ make

make install

cd ..

HDF5

wget -q http://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.1/src/hdf5-1.10.1.tar.gz tar xzf hdf5-1.10.1.tar.gz mv hdf5-1.10.1 hdf5 cd hdf5 && ./configure --enable-cxx --prefix=/usr make make install cd .. rm hdf5-1.10.1.tar.gz

sonLib

git clone https://github.com/ComparativeGenomicsToolkit/sonLib.git cd sonLib make cd ..

HAL

git clone https://github.com/ComparativeGenomicsToolkit/hal.git cd hal make

make install

cd ..

LibBigWig

git clone https://github.com/dpryan79/libBigWig.git cd libBigWig make

make install

cd ..

WiggleTools

git clone https://github.com/dahlo/WiggleTools ln -s /usr/bin/cc /bin/cc cd WiggleTools

make ... gives error below ... -lcrypto is missing from the cc command !

~/biotools/WiggleTools/src$ cc -g -Wall -O3 -std=gnu99 -L../lib -L../../libBigWig -L../../htslib wiggletools.c -lwiggletools -l:libBigWig.a -lcurl -l:libhts.a -lgsl -lgslcblas -lz -lpthread -lm -llzma -lbz2 -o ../bin/wiggletools

/usr/bin/ld: ../../htslib/libhts.a(hfile_s3.o): undefined reference to symbol 'EVP_sha256@@OPENSSL_3.0.0'

/usr/bin/ld: /lib/x86_64-linux-gnu/libcrypto.so.3: error adding symbols: DSO missing from command line

collect2: error: ld returned 1 exit status

Do this instead ...

mkdir -p ./bin cd src cc -g -Wall -O3 -std=gnu99 -L../lib -L../../libBigWig -L../../htslib wiggletools.c -lwiggletools -l:libBigWig.a -lcurl -l:libhts.a -lgsl -lgslcblas -lz -lpthread -lm -llzma -lbz2 -o ../bin/wiggletools -lcrypto cd .. make cd ..

sambamba

wget -q https://github.com/biod/sambamba/releases/download/v0.6.7/sambamba_v0.6.7_linux.tar.bz2 tar xvjf sambamba_v0.6.7_linux.tar.bz2 rm ./sambamba_v0.6.7_linux.tar.bz2

apt-get update apt-get install -y wget bedtools bamtools samtools sqlite3 libgsl0-dev libcolamd2 software-properties-common libcurl4-openssl-dev exonerate add-apt-repository -y ppa:deadsnakes/ppa apt-get install -y python3.7 python3-pip

for i in wigToBigWig faToTwoBit gff3ToGenePred genePredToBed genePredToFakePsl bamToPsl transMapPslToGenePred \ pslPosTarget axtChain chainMergeSort pslMap pslRecalcMatch pslMapPostChain gtfToGenePred genePredToGtf bedtools \ pslCheck pslCDnaFilter clusterGenes pslToBigPsl bedSort bedToBigBed; do \ wget -q http://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/$i -O /bin/$i ; chmod +x /bin/$i ; done

cp ./hal/bin/ /bin/ cp ./sambamba /bin/ cp ./augustus/bin/ /bin/ cp ./augustus/scripts/ /bin/ cp ./WiggleTools/bin/ /bin/

mkdir -p /augustus cp /augustus/config /augustus/config

Python deps

pip3 install bd2k-python-lib toil[all]==5.0 pyfasta numpy matplotlib pandas==1.0

make Python 3 primary python

rm /usr/bin/python ln -s /usr/bin/python3.7 /usr/bin/python

ENV AUGUSTUS_CONFIGPATH=/augustus/config/

[sgiorgetti]

Hello @marcovth - I have tried for since your message above, but could not reproduce the error; hence, unsure how to help.

I used Docker - not Singularity - and tried with ubuntu:20.04, bullseye, python:3.9-bullseye, and python:3.9-slime-bullseye. Also compiled samtools packages (htslib, bcftools, samtools) from both direct git clone and version-specific TAR archives.

In the end, I stumbled upon a few errors in the entire process of making the CAT Dockerfile work with Ubuntu 20.04 or Debian 11 ... but they seem to be more related to the multi-stage image building process "only", rather than real compatibility problems. And never stumbled upon the missing libcrypto error.

Also, abandoning any optimisation, I "translated" your steps above into yet another Dockerfile (started from python:3.9-bullseye). And - again - I could not get the "libcrypto" error.

[sgiorgetti]

Closing for now (not reproducible). @marcovth please feel free to reach out, should you need it.