linking error in zlibWrapper

[osuchanglab]

I have cloned the repository recursively and am trying to compile dashing using gcc 8.2.0 on centos linux. It compiles for a while but dies with an error while linking zlibWrapper:

g++ -O3 -funroll-loops -pipe -fno-strict-aliasing -DUSE_PDQSORT -DNOT_THREADSAFE -mpopcnt -flto -fopenmp -fno-rtti -std=c++14 -Wall -Wextra -Wno-char-subscripts -Wpointer-arith -Wwrite-strings -Wdisabled-optimization -Wformat -Wcast-align -Wno-unused-function -Wno-unused-parameter -pedantic -Wunused-variable -Wno-attributes -Wno-pedantic -Wno-ignored-attributes -Wno-missing-braces -Wno-unknown-pragmas -DDASHING_VERSION=\"v0.4.6\" -fdiagnostics-color=always -Ibonsai/clhash/include -I. -Ibonsai/zlib -Ibonsai/libpopcnt -Iinclude -Ibonsai/circularqueue -Ibonsai/zstd/zlibWrapper -Ibonsai/zstd/lib/common -Ibonsai/zstd/lib -Isketch/vec -Ibonsai -Ibonsai/include/ -Isketch/include -Isketch -Isketch/include/sketch -Isketch/vec -L. bonsai/zstd/zlibWrapper/gzclose.o bonsai/zstd/zlibWrapper/gzlib.o bonsai/zstd/zlibWrapper/gzread.o bonsai/zstd/zlibWrapper/gzwrite.o bonsai/zstd/zlibWrapper/zstd_zlibwrapper.o libzstd.a bonsai/clhash.o bonsai/klib/kthread.o src/main.o src/union.o src/hllmain.o src/mkdistmain.o src/finalizers.o src/cardests.o src/distmain.o src/construct.o src/flatten_all.o src/sketchcmpbbmh.o src/sketchcmpkhs.o src/sketchcmprmh.o src/sketchcmpcrmh.o src/sketchcmphll.o src/sketchcmpbf.o src/sketchcmpsmh.o src/sketchcorekhs.o src/sketchcorecbbmh.o src/sketchcoresmh.o src/sketchcorebbmh.o src/sketchcorermh.o src/sketchcorebf.o src/sketchcorehll.o src/sketchcorecrmh.o src/background.o -O3 src/dashing.o -o dashing -DZWRAP_USE_ZSTD=1 -lzstd -lz -ldl -march=native -DNDEBUG # -DNDEBUG bonsai/zstd/zlibWrapper/zstd_zlibwrapper.o: In function z_inflateGetDictionary': zstd_zlibwrapper.c:(.text+0x1ab9): undefined reference toinflateGetDictionary' collect2: error: ld returned 1 exit status make: *** [dashing] Error 1

I also tried running each of the three pre-compiled versions of dashing on a machine with AVX2 but each binary dies with a core dump: Dashing version: v0.4.5-8-g370d terminate called recursively terminate called recursively run_dashing.sh: line 6: 39845 Aborted (core dumped) ./dashing_s128 cmp -k 31 -p 8 -O distance_matrix.txt -o size_estimates.txt -F inputfiles.txt

[dnbaker]

Hi osuchanglab,

I had removed the zlib repository from bonsai to reduce the number of dependencies, but unfortunately, on systems with older zlib distributions, this causes problems. I'll patch that up as soon as I can. (In the meantime, if you clone and link against the official gzip, it should work.)

Regarding the linux builds, I'll see what I can find there.

Thanks for the bug report!

Daniel

[dnbaker]

Hi,

Just to update, I've corrected the compilation issue in [https://github.com/dnbaker/dashing/commit/a100d189d9eb27421a4a130ce9f49897ad55ee93](this pull request) by which I build directly with recent zlib.

Regarding the other issue, can you run your experiment with the pre-compiled binary with only 1 thread? terminate called recursively usually gets emitted when there's a runtime error encountered by multiple threads, so sometimes you'll need to run the program with a single thread to get an informative error message.

Thanks,

Daniel

[osuchanglab]

Hi Daniel,

Thank you for the response. I tried cloning zlib from that repository and was able to install it directly, however I still get the same error compiling dashing, with a few new errors as well:

bonsai/zstd/zlibWrapper/zstd_zlibwrapper.o: In function z_inflateGetDictionary': zstd_zlibwrapper.c:(.text+0x1ab9): undefined reference toinflateGetDictionary' bonsai/zstd/zlibWrapper/zstd_zlibwrapper.o: In function z_adler32_z': zstd_zlibwrapper.c:(.text+0x1ff1): undefined reference toadler32_z' bonsai/zstd/zlibWrapper/zstd_zlibwrapper.o: In function z_crc32_z': zstd_zlibwrapper.c:(.text+0x2001): undefined reference tocrc32_z' collect2: error: ld returned 1 exit status

I see that the way I cloned the dashing repository also cloned the latest zlib from github and installed with that, so I think the problem is still there. I cloned dashing with this command:

git clone --recursive --single-branch --branch v0.4.6 https://github.com/dnbaker/dashing.

which also cloned zlib and zstd and a few other things, as recommended in issue #35.

I apologize, I accidentally used our lab github instead of my own to post this. Thanks again!

Best, Alex

[osuchanglab]

Hi,

Edit: my previous comment was still using the older cloned github of dashing and did not include your newest commit.

Just a follow up since I noticed your comment on using a single thread. I tried that with the latest release (dashing_s128) and it runs correctly. Thanks!

Best, Alex

[dnbaker]

Hi Alex --

Thanks for checking in. The single-threaded suggestion was because gcc will often say "terminate called recursively" instead of emitting a message that can be used for debugging, though in normal usage, no exceptions should be thrown and you should be fine to use as many threads as you want.

You can access this either with the prebuilt binaries (here for the new ones) or building from master, where you shouldn't have any problems linking with zlib.

Let me know if you have any more problems or questions. I'm closing this for now, but feel free to reopen it as necessary.

Thanks!

Daniel