gmarcais / Jellyfish

A fast multi-threaded k-mer counter
Other
469 stars 136 forks source link

"make" fails when bindings turned on during configure #111

Closed transcriptomics closed 6 years ago

transcriptomics commented 6 years ago

[trans@s15-37 jellyfish-2.2.9]$ ./configure --prefix=/home/trans/apps/jellyfishv2.2.9 --enable-ruby-binding --enable-perl-binding --enable-python-binding checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /usr/bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether make supports nested variables... (cached) yes checking how to print strings... printf checking for style of include used by make... GNU checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether gcc understands -c and -o together... yes checking dependency style of gcc... gcc3 checking for a sed that does not truncate output... /usr/bin/sed checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for fgrep... /usr/bin/grep -F checking for ld used by gcc... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 25165824 checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for ar... ar checking for archiver @FILE support... @ checking for strip... strip checking for ranlib... ranlib checking command to parse /usr/bin/nm -B output from gcc object... ok checking for sysroot... no checking for a working dd... /usr/bin/dd checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 checking for mt... no checking if : is a manifest tool... no checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... no checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... no checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking for ld used by gcc... /usr/bin/ld -m elf_x86_64 checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes checking for shared library run path origin... done checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking how to run the C++ preprocessor... g++ -E checking for ld used by g++... /usr/bin/ld -m elf_x86_64 checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking for g++ option to produce PIC... -fPIC -DPIC checking if g++ PIC flag -fPIC -DPIC works... yes checking if g++ static flag -static works... no checking if g++ supports -c -o file.o... yes checking if g++ supports -c -o file.o... (cached) yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking dynamic linker characteristics... (cached) GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking for htslib... yes checking for samtools... no checking for md5sum... md5sum checking for yaggo... false checking for x86 cpuid output... unknown checking for x86-AVX xgetbv output... unknown checking for x86 cpuid 0x00000000 output... f:756e6547:6c65746e:49656e69 checking for x86 cpuid 0x80000000 output... 80000008:0:0:0 checking for x86 cpuid 0x00000001 output... 306f2:3100800:7dfefbff:bfebfbff checking for x86 cpuid 0x00000007 output... 0:37ab:0:0 checking for x86 cpuid 0x80000001 output... 0:0:21:2c100800 checking for x86-AVX xgetbv 0x00000000 output... 7:0 checking whether RDRND is supported by the processor... yes checking whether RDRND is supported by the processor and OS... yes checking whether C++ compiler accepts -mrdrnd... yes checking whether BMI1 is supported by the processor... yes checking whether BMI1 is supported by the processor and OS... yes checking whether C++ compiler accepts -mbmi... yes checking whether BMI2 is supported by the processor... yes checking whether BMI2 is supported by the processor and OS... yes checking whether C++ compiler accepts -mbmi2... yes checking whether ADX is supported by the processor... no checking whether MPX is supported by the processor... no checking whether PREFETCHWT1 is supported by the processor... no checking whether ABM is supported by the processor... yes checking whether ABM is supported by the processor and OS... yes checking whether C++ compiler accepts -mabm... yes checking whether MMX is supported by the processor... yes checking whether MMX is supported by the processor and OS... yes checking whether C++ compiler accepts -mmmx... yes checking whether SSE is supported by the processor... yes checking whether SSE is supported by the processor and OS... yes checking whether C++ compiler accepts -msse... yes checking whether SSE2 is supported by the processor... yes checking whether SSE2 is supported by the processor and OS... yes checking whether C++ compiler accepts -msse2... yes checking whether SSE3 is supported by the processor... yes checking whether SSE3 is supported by the processor and OS... yes checking whether C++ compiler accepts -msse3... yes checking whether SSSE3 is supported by the processor... yes checking whether SSSE3 is supported by the processor and OS... yes checking whether C++ compiler accepts -mssse3... yes checking whether SSE4.1 is supported by the processor... yes checking whether SSE4.1 is supported by the processor and OS... yes checking whether C++ compiler accepts -msse4.1... yes checking whether SSE4.2 is supported by the processor... yes checking whether SSE4.2 is supported by the processor and OS... yes checking whether C++ compiler accepts -msse4.2... yes checking whether SSE4a is supported by the processor... no checking whether SHA is supported by the processor... no checking whether AES is supported by the processor... no checking whether AVX is supported by the processor... yes checking whether AVX is supported by the processor and OS... yes checking whether C++ compiler accepts -mavx... yes checking whether FMA3 is supported by the processor... yes checking whether FMA3 is supported by the processor and OS... yes checking whether C++ compiler accepts -mfma... yes checking whether FMA4 is supported by the processor... no checking whether XOP is supported by the processor... no checking whether AVX2 is supported by the processor... yes checking whether AVX2 is supported by the processor and OS... yes checking whether C++ compiler accepts -mavx2... yes checking whether AVX512-F is supported by the processor... no checking whether AVX512-CD is supported by the processor... no checking whether AVX512-PF is supported by the processor... no checking whether AVX512-ER is supported by the processor... no checking whether AVX512-VL is supported by the processor... no checking whether AVX512-BW is supported by the processor... no checking whether AVX512-DQ is supported by the processor... no checking whether AVX512-IFMA is supported by the processor... no checking whether AVX512-VBMI is supported by the processor... no checking for __int128... yes checking for std::numeric_limits<__int128>... no checking for _NSGetExecutablePath... no checking for execinfo.h... yes checking for ext/stdio_filebuf.h... yes checking for sys/syscall.h... yes checking for siginfo_t.si_int... yes checking for python... /home/trans/sysapps/Pythonv3.6.4/bin/python checking for a version of Python >= '2.1.0'... yes checking for the distutils Python package... yes checking for Python include path... -I/home/trans/sysapps/Pythonv3.6.4/include/python3.6m checking for Python library path... -L/home/trans/sysapps/Pythonv3.6.4/lib -lpython3.6m checking for Python site-packages path... /home/trans/apps/jellyfishv2.2.9/lib/python3.6/site-packages checking python extra libraries... -lpthread -ldl -lutil -lm checking python extra linking flags... -Xlinker -export-dynamic checking consistency of all components of python development environment... yes checking for ruby... /home/trans/sysapps/rubyv2.5.0/bin/ruby checking for Ruby version... 2.5.0 checking for Ruby extensions target directory... /home/trans/apps/jellyfishv2.2.9/lib/ruby/2.5.0 checking for Ruby include directory... -I/home/trans/sysapps/rubyv2.5.0/include/ruby-2.5.0 -I/home/trans/sysapps/rubyv2.5.0/include/ruby-2.5.0/x86_64-linux checking for Ruby libs... -Wl,-rpath,/home/trans/sysapps/rubyv2.5.0/lib -L/home/trans/sysapps/rubyv2.5.0/lib -lruby -lpthread -lgmp -ldl -lcrypt -lm checking for Ruby extra LDFLAGS... checking for perl... /home/trans/sysapps/perlv5.26.1/bin/perl checking for Perl prefix... /home/trans/sysapps/perlv5.26.1 checking for Perl extension include path... /home/trans/sysapps/perlv5.26.1/lib/5.26.1/x86_64-linux-thread-multi/CORE checking for Perl extension target directory... /home/trans/apps/jellyfishv2.2.9/lib/perl/5.26.0 checking for Perl extensions C preprocessor flags... -D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include checking for Perl extensions linker flags... -shared -O2 -L/usr/local/lib -fstack-protector-strong checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating Makefile config.status: creating tests/compat.sh config.status: creating jellyfish-2.0.pc config.status: creating config.h config.status: executing depfiles commands config.status: executing libtool commands [trans@s15-37 jellyfish-2.2.9]$ make
make all-am make[1]: Entering directory `/home/trans/apps/jellyfish-2.2.9\' CXX lib/rectangular_binary_matrix.lo CXX lib/mer_dna.lo CXX lib/storage.lo CXX lib/allocators_mmap.lo CXX lib/misc.lo CXX lib/int128.lo CXX lib/thread_exec.lo CXX lib/jsoncpp.lo CXX lib/time.lo CXX lib/generator_manager.lo CXXLD libjellyfish-2.0.la CXX swig/perl5/swig_perl5_jellyfish_la-swig_wrap.lo CXXLD swig/perl5/jellyfish.la CXX swig/python/swig_python__dna_jellyfish_la-swig_wrap.lo CXXLD swig/python/_dna_jellyfish.la CXX swig/ruby/swig_ruby_jellyfish_la-swig_wrap.lo CXXLD swig/ruby/jellyfish.la CXX sub_commands/jellyfish.o CXX sub_commands/count_main.o CXX sub_commands/info_main.o CXX sub_commands/dump_main.o CXX sub_commands/histo_main.o CXX sub_commands/stats_main.o CXX sub_commands/merge_main.o CXX sub_commands/bc_main.o CXX sub_commands/query_main.o CXX sub_commands/cite_main.o CXX sub_commands/mem_main.o CXX jellyfish/mergefiles.o CXXLD bin/jellyfish PYTHONC swig/python/__init_\.pyc make[1]: No rule to make target `swig/python/jellyfish.py\', needed by `all-am\'. Stop. make[1]: Leaving directory `/home/trans/apps/jellyfish-2.2.9\' make: [all] Error 2

If bindings are not turned on the programs compiles without any error!

transcriptomics commented 6 years ago

Only python-binding needs to be turned off for 'make' to succeed. May be binding not compatible with python3? Looks like I learned the hard way!

[trans@sd15-37 jellyfish-2.2.9]$ ./configure --prefix=/home/trans/apps/jellyfishv2.2.9 --enable-ruby-binding --enable-perl-binding --enable-swig --with-sse --with-int128 … … … [trans@smsx10srw-srcf-d15-37 jellyfish-2.2.9]$ make -j12 make all-am make[1]: Entering directory `/home/trans/apps/jellyfish-2.2.9\' CXX lib/rectangular_binary_matrix.lo CXX lib/mer_dna.lo CXX lib/storage.lo CXX lib/misc.lo CXX lib/allocators_mmap.lo CXX lib/thread_exec.lo CXX lib/int128.lo CXX lib/jsoncpp.lo CXX lib/time.lo CXX lib/generator_manager.lo CXX swig/perl5/swig_perl5_jellyfish_la-swig_wrap.lo CXX swig/ruby/swig_ruby_jellyfish_la-swig_wrap.lo CXX sub_commands/jellyfish.o CXX sub_commands/count_main.o CXX sub_commands/info_main.o CXX sub_commands/dump_main.o CXX sub_commands/histo_main.o CXX sub_commands/stats_main.o CXX sub_commands/merge_main.o CXX sub_commands/bc_main.o CXX sub_commands/query_main.o CXX sub_commands/cite_main.o CXX sub_commands/mem_main.o CXX jellyfish/merge_files.o CXXLD libjellyfish-2.0.la CXXLD swig/ruby/jellyfish.la CXXLD bin/jellyfish CXXLD swig/perl5/jellyfish.la make[1]: Leaving directory `/home/trans/apps/jellyfish-2.2.9\' [trans@s15-37 jellyfish-2.2.9]$

gmarcais commented 6 years ago

Your two commands are not the same. The second one has --enable-swig.

If you start from the distribution tarball, you only need --enable-python-binding and there is no dependency on SWIG.

If you start from the git repository, you need --enable-swig in addition to --enable-python-binding and SWIG 3 is a dependency.

Python3 should be supported.

transcriptomics commented 6 years ago

Thanks for your prompt reply. I am using the distribution: https://github.com/gmarcais/Jellyfish/releases/download/v2.2.9/jellyfish-2.2.9.tar.gz

Thanks for clarifying the '--enable-swig' option. I was trying the different options to see if they work.

I can confirm that I get the same error as above when building the distribution release with or without the '--enable-swig' option.

transcriptomics commented 6 years ago

[SOLVED] Renaming the file dna_jellyfish.py solves the issue as below.

> [trans@s-d15-37 apps]$ tar xvf jellyfish-2.2.9.tar.gz
> …
> …
> [trans@s-d15-37 apps]$ cd jellyfish-2.2.9/
> [trans@s-d15-37 jellyfish-2.2.9]$ cp -av swig/python/dna_jellyfish.py swig/python/jellyfish.py     
> ‘swig/python/dna_jellyfish.py’ -> ‘swig/python/jellyfish.py’
> [trans@s-d15-37 jellyfish-2.2.9]$ ./configure --prefix=/home/trans/apps/jellyfishv2.2.9 --enable-ruby-binding --enable-perl-binding --with-sse --with-int128 --enable-python-binding
> …
> …
> [trans@s-d15-37 jellyfish-2.2.9]$ make -j12
> make  all-am
> make[1]: Entering directory `/home/trans/apps/jellyfish-2.2.9'
>   PYTHONC  swig/python/__init__.pyc
>   CXX      lib/rectangular_binary_matrix.lo
>   CXX      lib/allocators_mmap.lo
>   CXX      lib/mer_dna.lo
>   CXX      lib/storage.lo
>   CXX      lib/int128.lo
>   CXX      lib/misc.lo
>   CXX      lib/thread_exec.lo
>   CXX      lib/jsoncpp.lo
>   CXX      lib/time.lo
>   CXX      lib/generator_manager.lo
>   CXX      swig/perl5/swig_perl5_jellyfish_la-swig_wrap.lo
>   CXX      swig/python/swig_python__dna_jellyfish_la-swig_wrap.lo
>   CXX      swig/ruby/swig_ruby_jellyfish_la-swig_wrap.lo
>   CXX      sub_commands/jellyfish.o
>   CXX      sub_commands/count_main.o
>   CXX      sub_commands/info_main.o
>   CXX      sub_commands/dump_main.o
>   CXX      sub_commands/histo_main.o
>   CXX      sub_commands/stats_main.o
>   CXX      sub_commands/merge_main.o
>   CXX      sub_commands/bc_main.o
>   CXX      sub_commands/query_main.o
>   CXX      sub_commands/cite_main.o
>   CXX      sub_commands/mem_main.o
>   CXX      jellyfish/merge_files.o
>   CXXLD    libjellyfish-2.0.la
>   CXXLD    swig/python/_dna_jellyfish.la
>   CXXLD    swig/ruby/jellyfish.la
>   CXXLD    bin/jellyfish
>   CXXLD    swig/perl5/jellyfish.la
> make[1]: Leaving directory `/home/trans/apps/jellyfish-2.2.9'
> [trans@s-d15-37 jellyfish-2.2.9]$ make -j12 check
> make  check-am
> make[1]: Entering directory `/home/trans/apps/jellyfish-2.2.9'
> make  libgtest.la libgtest_main.la bin/generate_sequence bin/fastq2sam bin/test_all
> make[2]: Entering directory `/home/trans/apps/jellyfish-2.2.9'
>   CXX      jellyfish/generate_sequence.o
>   CXX      jellyfish/mersenne.o
>   CXX      jellyfish/dbg.o
>   CXX      jellyfish/backtrace.o
>   CXX      jellyfish/fastq2sam.o
>   CXX      jellyfish/bin_test_all-backtrace.o
>   CXX      unit_tests/gtest/src/libgtest_main_la-gtest_main.lo
>   CXX      unit_tests/gtest/src/libgtest_la-gtest-all.lo
>   CXX      unit_tests/bin_test_all-test_misc.o
>   CXX      unit_tests/bin_test_all-test_offsets_key_value.o
>   CXX      unit_tests/bin_test_all-test_simple_circular_buffer.o
>   CXX      unit_tests/bin_test_all-test_main.o
>   CXX      unit_tests/bin_test_all-test_rectangular_binary_matrix.o
>   CXX      unit_tests/bin_test_all-test_mer_dna.o
>   CXX      unit_tests/bin_test_all-test_large_hash_array.o
>   CXX      unit_tests/bin_test_all-test_mer_overlap_sequence_parser.o
>   CXX      unit_tests/bin_test_all-test_file_header.o
>   CXX      unit_tests/bin_test_all-test_mer_iterator.o
>   CXX      unit_tests/bin_test_all-test_hash_counter.o
>   CXX      unit_tests/bin_test_all-test_mer_heap.o
>   CXX      unit_tests/bin_test_all-test_stream_iterator.o
>   CXX      unit_tests/bin_test_all-test_token_ring.o
>   CXX      unit_tests/bin_test_all-test_text_dumper.o
>   CXX      unit_tests/bin_test_all-test_dumpers.o
>   CXX      unit_tests/bin_test_all-test_mapped_file.o
>   CXX      unit_tests/bin_test_all-test_int128.o
>   CXX      unit_tests/bin_test_all-test_mer_dna_bloom_counter.o
>   CXX      unit_tests/bin_test_all-test_whole_sequence_parser.o
>   CXX      unit_tests/bin_test_all-test_allocators_mmap.o
>   CXX      unit_tests/bin_test_all-test_cooperative_pool2.o
>   CXX      unit_tests/bin_test_all-test_generator_manager.o
>   CXX      unit_tests/bin_test_all-test_atomic_bits_array.o
>   CXX      unit_tests/bin_test_all-test_stdio_filebuf.o
>   CXXLD    bin/generate_sequence
>   CXXLD    bin/fastq2sam
>   CXXLD    libgtest.la
>   CXXLD    libgtest_main.la
>   CXXLD    bin/test_all
> make[2]: Leaving directory `/home/trans/apps/jellyfish-2.2.9'
> make  check-TESTS
> make[2]: Entering directory `/home/trans/apps/jellyfish-2.2.9'
> make[3]: Entering directory `/home/trans/apps/jellyfish-2.2.9'
> SKIP: tests/big.sh
> PASS: tests/generate_sequence.sh
> PASS: tests/small_mers.sh
> PASS: tests/swig_python.sh
> PASS: tests/subset_hashing.sh
> PASS: tests/bloom_filter.sh
> PASS: tests/bloom_counter.sh
> PASS: tests/swig_perl.sh
> PASS: tests/sam.sh
> PASS: tests/multi_file.sh
> PASS: tests/large_key.sh
> PASS: tests/merge.sh
> PASS: tests/parallel_hashing.sh
> PASS: tests/swig_ruby.sh
> PASS: unit_tests/unit_tests.sh
> ============================================================================
> Testsuite summary for jellyfish 2.2.9
> ============================================================================
> # TOTAL: 15
> # PASS:  14
> # SKIP:  1
> # XFAIL: 0
> # FAIL:  0
> # XPASS: 0
> # ERROR: 0
> ============================================================================
> make[3]: Leaving directory `/home/trans/apps/jellyfish-2.2.9'
> make[2]: Leaving directory `/home/trans/apps/jellyfish-2.2.9'
> make[1]: Leaving directory `/home/trans/apps/jellyfish-2.2.9'
> [trans@s-d15-37 jellyfish-2.2.9]$
> 
gmarcais commented 6 years ago

Copying that file does "work", in some sense. The coming distribution tarball will contain jellyfish.py, which prints a warning that it is deprecated and one should use dna_jellyfish.py instead.

Thanks for the bug report.