search

gmarcais / Jellyfish

A fast multi-threaded k-mer counter
Other
474 stars 136 forks source link

Fails compile on POWER8: rectangular_binary_matrix.hpp impossible register constraint in 'asm' #63

Open jlost opened 8 years ago

jlost commented 8 years ago

Hi,

I am not able to compile Jellyfish on an IBM POWER8 system with gcc. It seems that the raw assembly instructions in rectangular_binary_matrix.hpp are not compatible -- to build and run successfully will at least require preprocessor directives to offer alternative code / instructions.

Is this assessment accurate, or is there some other problem that I'm missing?

Any insight would be appreciated.

build.sh

curl -O https://github.com/gmarcais/Jellyfish/releases/download/v2.2.4/jellyfish-2.2.4.tar.gz
tar xf jellyfish-2.2.4.tar.gz
cd jellyfish-2.2.4
./configure
make

Error

[u0017592@sys-81783 jellyfish-2.2.4]$ make
make  all-am
make[1]: Entering directory `/home/u0017592/projects/jellyfish-2.2.4'
  CXX      lib/rectangular_binary_matrix.lo
In file included from lib/rectangular_binary_matrix.cc:22:0:
./include/jellyfish/rectangular_binary_matrix.hpp: In member function 'uint64_t jellyfish::RectangularBinaryMatrix::times_sse(const T&) const [with T = long unsigned int*; uint64_t = long unsigned int]':
./include/jellyfish/rectangular_binary_matrix.hpp:297:24: error: impossible register constraint in 'asm'
         AND_XOR("0x30");
                        ^
./include/jellyfish/rectangular_binary_matrix.hpp:335:3: error: impossible register constraint in 'asm'
   }
   ^
./include/jellyfish/rectangular_binary_matrix.hpp:300:24: error: impossible register constraint in 'asm'
         AND_XOR("0x20");
                        ^
./include/jellyfish/rectangular_binary_matrix.hpp:335:3: error: impossible register constraint in 'asm'
   }
   ^
./include/jellyfish/rectangular_binary_matrix.hpp:303:24: error: impossible register constraint in 'asm'
         AND_XOR("0x10");
                        ^
./include/jellyfish/rectangular_binary_matrix.hpp:335:3: error: impossible register constraint in 'asm'
   }
   ^
./include/jellyfish/rectangular_binary_matrix.hpp:306:20: error: impossible register constraint in 'asm'
         AND_XOR("");
                    ^
./include/jellyfish/rectangular_binary_matrix.hpp:335:3: error: impossible register constraint in 'asm'
   }
   ^
./include/jellyfish/rectangular_binary_matrix.hpp:316:22: error: impossible register constraint in 'asm'
       AND_XOR("0x20");
                      ^
./include/jellyfish/rectangular_binary_matrix.hpp:335:3: error: impossible register constraint in 'asm'
   }
   ^
./include/jellyfish/rectangular_binary_matrix.hpp:320:22: error: impossible register constraint in 'asm'
       AND_XOR("0x10");
                      ^
./include/jellyfish/rectangular_binary_matrix.hpp:335:3: error: impossible register constraint in 'asm'
   }
   ^
./include/jellyfish/rectangular_binary_matrix.hpp:324:18: error: impossible register constraint in 'asm'
       AND_XOR("");
                  ^
./include/jellyfish/rectangular_binary_matrix.hpp:335:3: error: impossible register constraint in 'asm'
   }
   ^
./include/jellyfish/rectangular_binary_matrix.hpp:333:25: error: impossible register constraint in 'asm'
         : [acc]"x"(acc));
                         ^
make[1]: *** [lib/rectangular_binary_matrix.lo] Error 1
make[1]: Leaving directory `/home/u0017592/projects/jellyfish-2.2.4'
make: *** [all] Error 2
[u0017592@sys-81783 jellyfish-2.2.4]$
gmarcais commented 8 years ago

Can you try configuring like so:

./configure --without-sse

That said, it has seen little to no testing on this architecture, I don't know if it will work. Do let me know.

jlost commented 8 years ago

./configure --without-sse allowed me to make it just fine. After make I ran make check, but it looks like there are some failures. Based on these two failing unit tests, would you expect everything to still work as intended?

[u0017592@sys-82143 jellyfish-2.2.4]$ sudo make check
make  check-am
make[1]: Entering directory `/home/u0017592/projects/jellyfish-2.2.4'
make  libgtest.la libgtest_main.la bin/generate_sequence bin/test_all
make[2]: Entering directory `/home/u0017592/projects/jellyfish-2.2.4'
  CXX      unit_tests/gtest/src/libgtest_la-gtest-all.lo
  CXXLD    libgtest.la
  CXX      unit_tests/gtest/src/libgtest_main_la-gtest_main.lo
  CXXLD    libgtest_main.la
  CXX      jellyfish/generate_sequence.o
  CXX      jellyfish/mersenne.o
  CXX      jellyfish/backtrace.o
  CXX      jellyfish/dbg.o
  CXXLD    bin/generate_sequence
  CXX      unit_tests/bin_test_all-test_main.o
  CXX      unit_tests/bin_test_all-test_misc.o
  CXX      unit_tests/bin_test_all-test_offsets_key_value.o
  CXX      unit_tests/bin_test_all-test_simple_circular_buffer.o
  CXX      unit_tests/bin_test_all-test_rectangular_binary_matrix.o
  CXX      unit_tests/bin_test_all-test_mer_dna.o
  CXX      unit_tests/bin_test_all-test_large_hash_array.o
  CXX      unit_tests/bin_test_all-test_mer_overlap_sequence_parser.o
  CXX      unit_tests/bin_test_all-test_file_header.o
  CXX      unit_tests/bin_test_all-test_mer_iterator.o
  CXX      unit_tests/bin_test_all-test_hash_counter.o
  CXX      unit_tests/bin_test_all-test_mer_heap.o
  CXX      unit_tests/bin_test_all-test_stream_iterator.o
  CXX      unit_tests/bin_test_all-test_token_ring.o
  CXX      unit_tests/bin_test_all-test_text_dumper.o
  CXX      unit_tests/bin_test_all-test_dumpers.o
  CXX      unit_tests/bin_test_all-test_mapped_file.o
  CXX      unit_tests/bin_test_all-test_int128.o
  CXX      unit_tests/bin_test_all-test_mer_dna_bloom_counter.o
  CXX      unit_tests/bin_test_all-test_whole_sequence_parser.o
  CXX      unit_tests/bin_test_all-test_allocators_mmap.o
  CXX      unit_tests/bin_test_all-test_cooperative_pool2.o
  CXX      unit_tests/bin_test_all-test_generator_manager.o
  CXX      unit_tests/bin_test_all-test_atomic_bits_array.o
  CXX      unit_tests/bin_test_all-test_stdio_filebuf.o
  CXX      jellyfish/bin_test_all-backtrace.o
  CXXLD    bin/test_all
make[2]: Leaving directory `/home/u0017592/projects/jellyfish-2.2.4'
make  check-TESTS
make[2]: Entering directory `/home/u0017592/projects/jellyfish-2.2.4'
make[3]: Entering directory `/home/u0017592/projects/jellyfish-2.2.4'
PASS: tests/generate_sequence.sh
PASS: tests/parallel_hashing.sh
PASS: tests/merge.sh
PASS: tests/bloom_filter.sh
SKIP: tests/big.sh
PASS: tests/subset_hashing.sh
PASS: tests/multi_file.sh
PASS: tests/bloom_counter.sh
FAIL: tests/large_key.sh
SKIP: tests/swig_python.sh
SKIP: tests/swig_ruby.sh
SKIP: tests/swig_perl.sh
FAIL: unit_tests/unit_tests.sh
============================================================================
Testsuite summary for jellyfish 2.2.4
============================================================================
# TOTAL: 13
# PASS:  7
# SKIP:  4
# XFAIL: 0
# FAIL:  2
# XPASS: 0
# ERROR: 0
============================================================================
See ./test-suite.log
Please report to gmarcais@umd.edu
============================================================================
make[3]: *** [test-suite.log] Error 1
make[3]: Leaving directory `/home/u0017592/projects/jellyfish-2.2.4'
make[2]: *** [check-TESTS] Error 2
make[2]: Leaving directory `/home/u0017592/projects/jellyfish-2.2.4'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/home/u0017592/projects/jellyfish-2.2.4'
make: *** [check] Error 2
gmarcais commented 8 years ago

Can you send me the content of test-suite.log?

jlost commented 8 years ago

Sure, here you go: test-suite.log.txt

gmarcais commented 8 years ago

Short answer: I look at the log, and I am pretty confident that it is working as expected.

Long answer: the large_key.sh test fails not because of Jellyfish, but because 'time' is not installed on your system. Not the shell built-in function, but the program, usually in /usr/bin/time. You can remove the call to time in tests/test_large_key.sh and rerun:

make check TESTS=tests/test_large_key.sh

The unit testing error can be safely ignored. It occurs when using __int128 as the base type for storing mers, which is an experiment and not done by default. Everywhere else but in these few tests, uint64_t is used, and all the tests with uint64_t are successful. In addition the failure occurs in a method that is only used in the script binding, so it would not affect k-mer counting anyway.

jlost commented 8 years ago

Hi again @gmarcais,

I see two improvements could be made to have this build more cleanly:

a) --without-sse could be inferred when on ppc64le instead of needing to be explicitly supplied.

b) Some unit tests fail when they shouldn't.

If I created these as separate issues, would you be willing to accept pull requests for them?

gmarcais commented 8 years ago

Sure, I will consider your pull requests.

dmiller423 commented 7 years ago

pull req https://github.com/gmarcais/Jellyfish/pull/80

=====================================================================

Testsuite summary for jellyfish 2.2.6

TOTAL: 13

PASS: 9

SKIP: 4

XFAIL: 0

FAIL: 0

XPASS: 0

ERROR: 0

=====================================================================

ashu-22 commented 7 years ago

Still, there is any bounty for this!

rafaeldelucena commented 7 years ago

Hello @ashu-22, you can use this #68 PR and configure --without-128 (__int128 as the base type for storing mers, it is an experiment mentioned here) to successful pass all unit tests for JellyFish.