Can't build dashing

[tseemann]

After a huge recursive module fetching round, I get this:

In file included from bonsai/cppitertools/internal/iter_tuples.hpp:4:0,
                 from bonsai/cppitertools/product.hpp:4,
                 from src/testsat.cpp:8:
bonsai/cppitertools/internal/iterator_wrapper.hpp:6:19: fatal error: variant: No such file or direct
ory
compilation terminated.
make: *** [Makefile:100: testsat] Error 1
make: *** Waiting for unfinished jobs....
In file included from ./bonsai/hll/cbf.h:3:0,
                 from src/testcbf.cpp:2:
./bonsai/hll/bf.h: In member function 'void sketch::bf::bfbase_t<HashStruct>::reseed(uint64_t)':
./bonsai/hll/bf.h:110:31: error: expected ')' before ';' token
             if(auto val = mt(); std::find(seeds_.cbegin(), seeds_.cend(), val) == seeds_.cend())

[dnbaker]

It seems like your compiler isn't fully supporting C++17. <variant> is C++17, and the bonsai/hll/bf.h file is using C++17 the if (initialization statement; condition) syntax. It should be getting the -std=c++17 flag from the Makefile. Can you see if it's being passed this parameter as expected?

For reference, the command I ended up using was g++-8 -O3 -funroll-loops -pipe -fno-strict-aliasing -march=native -mpclmul -fopenmp -fno-rtti -std=c++17 -Wall -Wextra -Wno-char-subscripts -Wpointer-arith -Wwrite-strings -Wdisabled-optimization -Wformat -Wcast-align -Wno-unused-function -Wno-unused-parameter -pedantic -DUSE_PDQSORT -Wunused-variable -Ibonsai/clhash/include -I. -Ibonsai/zlib -Ibonsai/libpopcnt -Iinclude -Ibonsai/circularqueue -Ibonsai/zstd/zlibWrapper -Ibonsai/zstd/lib/common -Ibonsai/zstd/lib -I/usr/local/opt/zlib/include -Ibonsai/hll -Ibonsai/hll/vec -Ibonsai/pdqsort -Ibonsai -Ibonsai/bonsai/include/ -L. -Lbonsai/zlib -Lbonsai/zstd/lib bonsai/klib/kstring.o bonsai/klib/kthread.o bonsai/bonsai/clhash.o -DNDEBUG src/dash.cpp -o dash -lz

[tseemann]

Ok that makes sense. Currently on 5.5 but can force gcc-6.

Can you add this fact to the docs that it needs a C++17 compiler?

Looks like gcc-8 is needed for most features? https://www.gnu.org/software/gcc/projects/cxx-status.html#cxx17

Does it work with clang ?

[dnbaker]

I'll have to update the README, thank you for that.

I don't actually need all of them. I've built dash and bonsai with gcc-[6-8]. If I remember correctly, 6.2 is new enough. if constexpr and type_traits variable templates are the biggest things. It looks like it would be quite a lot of work to make all of these code bases C++14 compliant. I'm also working on a static build.

[ashishdamania]

I was able to install dashing after meeting requirements for C++14 using SCL: https://docs.fedoraproject.org/en-US/Fedora_Contributor_Documentation/1/html-single/Software_Collections_Guide/index.html#sect-What_Are_Software_Collections

yum search devtoolset

Install the appropriate one found on your system.

yum install devtoolset-7-gcc.x86_64

Enable the collections:

scl enable devtoolset-7 'bash'

execute bash bash

Check if you can see the newer version of g++

g++ --version

Install the dashing git clone --recursive https://github.com/dnbaker/dashing cd dashing && make update dashing

I guess this works for Centos/Fedora or RHEL. Not sure if there are similar tools available for Ubuntu or other Linux distributions.

Thanks!

[dnbaker]

Checking in on this, we've walked back to be compatible with C++14, and we've tested with gcc{6-9} on OSX, Ubuntu, and CentOS. Would you be willing to give this another try?

Thanks!

[ashishdamania]

Hi Daniel, I tried using gcc version 6 using SCL as described above

[d@dcl-mbiome-p01 dashing]$ g++ --version
g++ (GCC) 6.3.1 20170216 (Red Hat 6.3.1-3)

Install dashing (latest version from Github)

git clone --recursive https://github.com/dnbaker/dashing
cd dashing && make update dashing

I am getting this warning and the subsequent error.

In file included from ./bonsai/bonsai/include/entropy.h:2:0,
                 from ./bonsai/bonsai/include/encoder.h:12,
                 from ./bonsai/bonsai/include/database.h:4,
                 from src/dashing.cpp:5:
bonsai/circularqueue/circular_buffer.h: In instantiation of ‘T circ::roundup(T) [with T = unsigned int]’:
bonsai/circularqueue/circular_buffer.h:147:26:   required from ‘circ::FastCircularQueue<T, SizeType>::FastCircularQueue(SizeType) [with T = bns::ElScore<long unsigned int, long unsigned int>; SizeType = unsigned int]’
./bonsai/bonsai/include/qmap.h:46:47:   required from ‘bns::QueueMap<T, ScoreType>::QueueMap(std::size_t) [with T = long unsigned int; ScoreType = long unsigned int; std::size_t = long unsigned int]’
./bonsai/bonsai/include/encoder.h:154:33:   required from ‘bns::Encoder<ScoreType>::Encoder(char*, bns::u64, const bns::Spacer&, void*, bool, uint64_t) [with ScoreType = bns::score::Entropy; bns::u64 = long unsigned int; uint64_t = long unsigned int]’
src/dashing.cpp:287:9:   required from here
bonsai/circularqueue/circular_buffer.h:30:37: warning: right shift count >= width of type [-Wshift-count-overflow]
     CIRC_CONSTIF(sizeof(T) > 4) x|=x>>32;
                                    ~^~~~
make: *** [dashing] Error 1

It seems to work fine for gcc version 7 (using SCL) for me.

Thanks for working on it.

[dnbaker]

Thanks for the testing! You should be able to safely ignore the circular buffer warning. (The compiler elides that shift before sizeof(T) <= 4; In C++17, the statement isn't even compiled unless the size of the type is > 4.)

Is there an error farther up in the log? make update dashing &> err might help you find the error, as sometimes the compiler emits warnings after the fatal error causing it.

[ashishdamania]

Here is the error file.

err.txt

[d@dcl-mbiome-p01 dashing]$ grep "error" err
bonsai/hll/common.h:201:30: error: ‘mullo’ is not a member of ‘sketch::common::Space {aka vec::SIMDTypes<long unsigned int>}’

Hope this helps.

[dnbaker]

Thank you!

There's some code for a template which the executable doesn't need which there was an error in, so I imagine some versions of the compiler omitted checking and some didn't.

Regardless, I've fixed this in this commit. Would you give it another shot?

[ashishdamania]

Now it seems to work ok. No errors and just warnings. Wow, that was quick! Thanks for working on it.