search

genome / pindel

Pindel can detect breakpoints of large deletions, medium sized insertions, inversions, tandem duplications and other structural variants at single-based resolution from next-gen sequence data. It uses a pattern growth approach to identify the breakpoints of these variants from paired-end short reads.
GNU General Public License v3.0
162 stars 89 forks source link

Pindel does not install with HTSLib - undefined reference to `hts_drand48()' #76

Closed jst43 closed 6 years ago

jst43 commented 6 years ago

I've installed htslib from git as:

git clone https://github.com/samtools/htslib.git
cd htslib/
autoheader
autoreconf
./configure
make
make install

which ran without errors. Now when I try to install pindel as:

git clone https://github.com/genome/pindel.git
cd pindel/
./INSTALL ../htslib/

I get the following error message:

path is now: /mnt/raid/Resources/Software/htslib/
make -C src pindel
make[1]: Entering directory '/mnt/raid/Resources/Software/pindel/src'
g++  -L/mnt/raid/Resources/Software/htslib/ -Wl,-rpath /mnt/raid/Resources/Software/htslib/ pindel.o reader.o reporter.o searcher.o parameter.o refreader.o control_state.o search_deletions_nt.o search_inversions.o search_inversions_nt.o bam2depth.o search_tandem_duplications.o search_tandem_duplications_nt.o output_sorter.o farend_searcher.o search_variant.o searchshortinsertions.o searchdeletions.o output_file_data.o bddata.o shifted_vector.o read_buffer.o line_reader.o ifstream_line_reader.o gz_line_reader.o pindel_read_reader.o user_defined_settings.o fn_parameters.o logstream.o search_MEI_util.o search_MEI.o assembly.o genotyping.o -O3 -fopenmp -lhts -lm -lz -o pindel
reader.o: In function `ks_shuffle_uint32_t(unsigned long, unsigned int*)':
/mnt/raid/Resources/Software/pindel/src/reader.cpp:67: undefined reference to `hts_drand48()'
collect2: error: ld returned 1 exit status
Makefile:36: recipe for target 'pindel' failed
make[1]: *** [pindel] Error 1
make[1]: Leaving directory '/mnt/raid/Resources/Software/pindel/src'
Makefile:12: recipe for target 'pindel' failed
make: *** [pindel] Error 2

mv: cannot stat 'src/pindel': No such file or directory
mv: cannot stat 'src/pindel2vcf': No such file or directory
mv: cannot stat 'src/sam2pindel': No such file or directory
mv: cannot stat 'src/pindel2vcf4tcga': No such file or directory

Pindel successfully compiled. The pindel executable can be found in this directory.

You can test pindel with the example files in the demo directory; example commands are given in the RUNME in that directory.

For help and background information, see the pindel wiki and its FAQ on gmt.genome.wustl.edu/packages/pindel, as well as
the FAQ file in the Pindel root directory (the same directory as this INSTALL script is located)
or contact us on kaiye@xjtu.edu.cn
jmarshall commented 6 years ago

This hts_drand48 malarky in ksort.h is a recent addition to htslib by @daviesrob.

What platform are you on? Has htslib been installed previously? Is it possible that you have compiled the Pindel .cpp files against headers in your ../htslib directory but this link command is picking up an older libhts.a?

jst43 commented 6 years ago

I'm running Ubuntu 16.04 LTS, I recently installed samtools 1.3.1 which comes with htslib-1.3.1, but I was unable to install Pindel using the path to that htslib directory (I think the error was the same one as in this issue.

I have two libhts.a files; one for the htslib that was installed with samtools and one for the htslib I downloaded from git. I don't know how I would check which copy of libhts.a is being picked up in the installation.

sarmadym commented 6 years ago

I am having the exact same issue on RHEL7

jmarshall commented 6 years ago

Check which one is being pulled in by adding -Wl,-map,pindel.map to the g++ -L … command, or more quickly by checking whether there are any libhts.{a,so*} files in /mnt/raid/Resources/Software/htslib (i.e., the directory that is the argument to -L).

Alternatively point Pindel's ./INSTALL at the location where you installed this htslib rather than your htslib source directory tree, though Pindel's homegrown configury is somewhat rustic and this may or may not work well.

PR #75 has the gist of a workaround, but it is a terrible hack rather than the right solution.

daviesrob commented 6 years ago

Yes, it should be picking up hts_drand48() from libhts. You could try using nm to see if the library you expect to have it actually does:

$ nm -a ../htslib/libhts.a | grep hts_drand48
                 U hts_drand48
                 U hts_drand48
00000000000001dc T hts_drand48
$ nm -a ../htslib/libhts.so | grep hts_drand48
00000000000357e8 T hts_drand48
jst43 commented 6 years ago

@jmarshall In the /mnt/raid/Resources/Software/htslib/ directory I have libhts.a, libhts.so, and libhts.so.2 @daviesrob Here's the outputs of the nm commands I ran:

$ nm ../htslib/libhts.a | grep hts_drand48
                 U hts_drand48
                 U hts_drand48
0000000000000150 T hts_drand48
$ nm ../htslib/libhts.so | grep hts_drand48
000000000002e480 T hts_drand48
daviesrob commented 6 years ago

Gah! The problem is that it's missing the extern "C" { } wrapper around the declaration of hts_drand48() in ksort.h. It works fine if you're using C, but not with C++ due to name mangling.

Applying this patch should fix it for you:

diff --git a/htslib/ksort.h b/htslib/ksort.h
index f50aa71..cf99884 100644
--- a/htslib/ksort.h
+++ b/htslib/ksort.h
@@ -65,6 +65,10 @@
 #include <stdlib.h>
 #include <string.h>

+#ifdef __cplusplus
+extern "C" {
+#endif
+
 // Use our own drand48() symbol (used by ks_shuffle) to avoid portability
 // problems on Windows.  Don't include htslib/hts_os.h for this as it
 // may not get on with older attempts to fix this in code that includes
@@ -289,4 +293,8 @@ typedef const char *ksstr_t;
 #define KSORT_INIT_GENERIC(type_t) KSORT_INIT(type_t, type_t, ks_lt_generic)
 #define KSORT_INIT_STR KSORT_INIT(str, ksstr_t, ks_lt_str)

+#ifdef __cplusplus
+}
+#endif
+
 #endif

I'll apply this to htslib and check it in.

jst43 commented 6 years ago

Can confirm it's working for me now, thanks!

jmarshall commented 6 years ago

D'oh!

There's an extent to which you only want extern "C" around the new extern declaration, not the bulk of ksort.h.

jst43 commented 6 years ago

@jmarshall so should the } be placed after extern double hts_drand48(void); instead of at the end of the file?

daviesrob commented 6 years ago

@jmarshall True, but I don't think it really matters, does it? And putting the closing bracket at the end means it's more likely to catch any similar problems in the future (not that I think there will be any, as I don't think we'd want to declare anything else in there).

oeco28 commented 5 years ago

I am still having a problem with this. I tried installing pindel with htslib_1.8. I can see that ksort.h has a extern "C" { ... statement around the declaration of drand48() so I am not sure what the problem might be. I run within the pindel folder the command: ./INSTALL /path_to_htslib/

and I get the following error:

make -C src pindel make[1]: Entering directory '/home/repository_software/pindel/src' g++ -I/home/repository_software/htslib-1.8 -Wall -g -c -O3 -fopenmp pindel.cpp -o pindel.o pindel.cpp: In function ‘void MergeInterChr(ControlState&, UserDefinedSettings*)’: pindel.cpp:1557:66: error: call of overloaded ‘abs(unsigned int)’ is ambiguous if (abs(All[index_a].FirstPos - All[index_b].FirstPos) < 10 && abs(All[index_a].SecondPos - All[index_b].SecondPos) < 10 && All[index_a].NumSupport + All[index_b].NumSupport >= cutoff) { ^ In file included from /usr/include/c++/7/cstdlib:75:0, from /usr/include/c++/7/bits/stl_algo.h:59, from /usr/include/c++/7/algorithm:62, from pindel.cpp:23: /usr/include/stdlib.h:837:12: note: candidate: int abs(int) extern int abs (int x) THROW attribute ((const)) wur; ^~~ In file included from /usr/include/c++/7/cstdlib:77:0, from /usr/include/c++/7/bits/stl_algo.h:59, from /usr/include/c++/7/algorithm:62, from pindel.cpp:23: /usr/include/c++/7/bits/std_abs.h:56:3: note: candidate: long int std::abs(long int) abs(long i) { return builtin_labs(__i); } ^~~ /usr/include/c++/7/bits/std_abs.h:61:3: note: candidate: long long int std::abs(long long int) abs(long long x) { return builtin_llabs (__x); } ^~~ /usr/include/c++/7/bits/std_abs.h:70:3: note: candidate: constexpr double std::abs(double) abs(double x) ^~~ /usr/include/c++/7/bits/std_abs.h:74:3: note: candidate: constexpr float std::abs(float) abs(float x) ^~~ /usr/include/c++/7/bits/std_abs.h:78:3: note: candidate: constexpr long double std::abs(long double) abs(long double __x) ^~~ /usr/include/c++/7/bits/std_abs.h:84:3: note: candidate: constexpr int128 std::abs(int128) abs(GLIBCXX_TYPE_INT_N_0 x) { return x >= 0 ? x : -x; } ^~~ /usr/include/c++/7/bits/std_abs.h:102:3: note: candidate: constexpr float128 std::abs(float128) abs(float128 x) ^~~ pindel.cpp:1557:127: error: call of overloaded ‘abs(unsigned int)’ is ambiguous (abs(All[index_a].FirstPos - All[index_b].FirstPos) < 10 && abs(All[index_a].SecondPos - All[index_b].SecondPos) < 10 && All[index_a].NumSupport + All[index_b].NumSupport >= cutoff) { ^ In file included from /usr/include/c++/7/cstdlib:75:0, from /usr/include/c++/7/bits/stl_algo.h:59, from /usr/include/c++/7/algorithm:62, from pindel.cpp:23: /usr/include/stdlib.h:837:12: note: candidate: int abs(int) extern int abs (int x) THROW attribute ((const)) wur; ^~~ In file included from /usr/include/c++/7/cstdlib:77:0, from /usr/include/c++/7/bits/stl_algo.h:59, from /usr/include/c++/7/algorithm:62, from pindel.cpp:23: /usr/include/c++/7/bits/std_abs.h:56:3: note: candidate: long int std::abs(long int) abs(long i) { return builtin_labs(__i); } ^~~ /usr/include/c++/7/bits/std_abs.h:61:3: note: candidate: long long int std::abs(long long int) abs(long long x) { return builtin_llabs (__x); } ^~~ /usr/include/c++/7/bits/std_abs.h:70:3: note: candidate: constexpr double std::abs(double) abs(double x) ^~~ /usr/include/c++/7/bits/std_abs.h:74:3: note: candidate: constexpr float std::abs(float) abs(float x) ^~~ /usr/include/c++/7/bits/std_abs.h:78:3: note: candidate: constexpr long double std::abs(long double) abs(long double __x) ^~~ /usr/include/c++/7/bits/std_abs.h:84:3: note: candidate: constexpr int128 std::abs(int128) abs(GLIBCXX_TYPE_INT_N_0 x) { return x >= 0 ? x : -x; } ^~~ /usr/include/c++/7/bits/std_abs.h:102:3: note: candidate: constexpr float128 std::abs(float128) abs(float128 x) ^~~ Makefile:47: recipe for target 'pindel.o' failed make[1]: [pindel.o] Error 1 make[1]: Leaving directory '/home/repository_software/pindel/src' Makefile:12: recipe for target 'pindel' failed make: [pindel] Error 2

mv: cannot stat 'src/pindel': No such file or directory mv: cannot stat 'src/pindel2vcf': No such file or directory mv: cannot stat 'src/sam2pindel': No such file or directory mv: cannot stat 'src/pindel2vcf4tcga': No such file or directory

Pindel successfully compiled. The pindel executable can be found in this directory.

You can test pindel with the example files in the demo directory; example commands are given in the RUNME in that directory.

For help and background information, see the pindel wiki and its FAQ on gmt.genome.wustl.edu/packages/pindel, as well as the FAQ file in the Pindel root directory (the same directory as this INSTALL script is located) or contact us on kaiye@xjtu.edu.cn

I also tried installing it with samtools_1.3 but I am not having much luck either. Have you seen this happen?

thank you, Omar

jmarshall commented 5 years ago

The abs problem is fixed by applying #64.

oeco28 commented 5 years ago

I am not sure this is the problem. This is how ksort.h statement in htslib 1.8 is:

ifdef __cplusplus

extern "C" {

endif

// Use our own drand48() symbol (used by ks_shuffle) to avoid portability

// problems on Windows. Don't include htslib/hts_os.h for this as it

// may not get on with older attempts to fix this in code that includes

// this file.

extern double hts_drand48(void);

typedef struct {

    void *left, *right;

    int depth;

} ks_isort_stack_t;

I am not sure that declaring extern "C" { prior to hts_drand48(void) is the problem. I might be understanding something wrong.

Omar

On Thu, Aug 2, 2018 at 2:29 AM John Marshall notifications@github.com wrote:

The abs problem is fixed by applying #64 https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_genome_pindel_pull_64&d=DwMCaQ&c=C3yme8gMkxg_ihJNXS06ZyWk4EJm8LdrrvxQb-Je7sw&r=1YnLN6ptmFumCtKMs62AHQdWF0mxKnIbu0uh8D6U1rE&m=avBfWLS-WVNRXy6ffwe51Zf3MRYJqcvN8rUfSKcAt3I&s=Rrx4la6eA3wjq38ZXxOJVMAE_kAgR2OX7XgJFu5znUc&e= .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_genome_pindel_issues_76-23issuecomment-2D409865507&d=DwMCaQ&c=C3yme8gMkxg_ihJNXS06ZyWk4EJm8LdrrvxQb-Je7sw&r=1YnLN6ptmFumCtKMs62AHQdWF0mxKnIbu0uh8D6U1rE&m=avBfWLS-WVNRXy6ffwe51Zf3MRYJqcvN8rUfSKcAt3I&s=2Wxp_aBbIvlFckTsBQzqxzBeubi-Ze4LeYjNFT2LhdY&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ANrYaw4LXWgvjbWyJV8lj9D-5FOY-2Do6c-2Drks5uMsZvgaJpZM4QQ-5FEI&d=DwMCaQ&c=C3yme8gMkxg_ihJNXS06ZyWk4EJm8LdrrvxQb-Je7sw&r=1YnLN6ptmFumCtKMs62AHQdWF0mxKnIbu0uh8D6U1rE&m=avBfWLS-WVNRXy6ffwe51Zf3MRYJqcvN8rUfSKcAt3I&s=C50t1SxQSGOjMtmGpIR2iLzl6jptg0VQ2LvQokmnMZc&e= .

-- Omar E. Cornejo, PhD Assistant Professor, School of Biological Sciences Affiliate Faculty Paul G. Allen School for Global Animal Health Washington State University Heald 429. PO Box 644236 Pullman, WA. 99164-4236