Running xxhash tests

[pengwinsurf]

Hi,

I am trying to run

make test-all

but I am getting an error

cli/xsum_sanity_check.c:360:15: runtime error: call to function XXH3_64bits_update through pointer to incorrect function type 'XXH_errorcode (*)(struct XXH3_state_s *, const void *, unsigned long)'
/home/ubuntu/xxhash/./xxhash.h:6339: note: XXH3_64bits_update defined here

Not sure why that is the case or potential reason for it ?

[t-mat]

Hi @pengwinsurf, Thank you for the report.

Honestly, we have no idea about this error, since our CI invokes make test-all for each single commit. But it also doesn't mean that our CI is perfect.

Therefore, we need an investigation.

It's tedious, but could you run the following commands, and put their entire log to this issue?

1. Checking environment

First of all, please check and report your environment.

uname -a
cat /etc/*-release # showing distro name
cc --version
make --version
git --version

2. make test-all

Please note : For investigation, we need entire log of make test-all.

cd
git clone --depth 1 --branch v0.8.2 https://github.com/Cyan4973/xxHash.git xxh-issue-927
cd xxh-issue-927/
git log -1
make test-all

[pengwinsurf]

uname -a
Linux venus 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"
PRETTY_NAME="Ubuntu 22.04.3 LTS"

cc --version
cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

 git --version
git version 2.34.1

commit bbb27a5efb85b92a0486cf361a8635715a53f6ba (grafted, HEAD, tag: v0.8.2)
Author: Yann Collet <Cyan4973@users.noreply.github.com>
Date:   Fri Jul 21 03:01:35 2023 -0700

    Merge pull request #873 from Cyan4973/dev

    v0.8.2

---- check undefined behavior - sanitize ----
make clean
make[1]: Entering directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927'
make -C tests clean
make[2]: Entering directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927/tests'
make[2]: Leaving directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927/tests'
make -C tests/bench clean
make[2]: Entering directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927/tests/bench'
rm -f *.o benchHash benchHash32 benchHash_avx2 benchHash_hw
make[2]: Leaving directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927/tests/bench'
make -C tests/collisions clean
make[2]: Entering directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927/tests/collisions'
rm -f *.o allcodecs/*.o
rm -f collisionsTest
make[2]: Leaving directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927/tests/collisions'
cleaning completed
make[1]: Leaving directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927'
make test CC=clang CXX=clang++ MOREFLAGS="-g -fsanitize=undefined -fno-sanitize-recover=all"
make[1]: Entering directory '/home/ubuntu/apiusage/data/libraries/xxh-issue-927'
clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsanitize=undefined -fno-sanitize-recover=all   -c -o xxhash.o xxhash.c
ar rcs libxxhash.a xxhash.o
clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsanitize=undefined -fno-sanitize-recover=all -fPIC  xxhash.c -shared -Wl,-soname=libxxhash.so.0 -o libxxhash.so.0.8.2
ln -sf libxxhash.so.0.8.2 libxxhash.so.0
ln -sf libxxhash.so.0.8.2 libxxhash.so
clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsanitize=undefined -fno-sanitize-recover=all   -c -o cli/xxhsum.o cli/xxhsum.c
clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsanitize=undefined -fno-sanitize-recover=all   -c -o cli/xsum_os_specific.o cli/xsum_os_specific.c
clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsa
nitize=undefined -fno-sanitize-recover=all   -c -o cli/xsum_output.o cli/xsum_output.c
clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsanitize=undefined -fno-sanitize-recover=all   -c -o cli/xsum_sanity_check.o cli/xsum_sanity_check.c
clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsanitize=undefined -fno-sanitize-recover=all   -c -o cli/xsum_bench.o cli/xsum_bench.c
clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsanitize=undefined -fno-sanitize-recover=all  xxhash.o cli/xxhsum.o cli/xsum_os_specific.o cli/xsum_output.o cli/xsum_sanity_check.o cli/xsum_bench.o  -o xxhsum
/usr/bin/ld: cannot find /usr/local/lib/clang/17/lib/linux/libclang_rt.ubsan_standalone-x86_64.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[t-mat]

Thanks for the info. But it seems your actual error:

clang -O3 -Wall -Wextra -Wconversion -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wformat-security -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wstrict-overflow=2 -DXXH_DEBUGLEVEL=1 -g -fsanitize=undefined -fno-sanitize-recover=all  xxhash.o cli/xxhsum.o cli/xsum_os_specific.o cli/xsum_output.o cli/xsum_sanity_check.o cli/xsum_bench.o  -o xxhsum
/usr/bin/ld: cannot find /usr/local/lib/clang/17/lib/linux/libclang_rt.ubsan_standalone-x86_64.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

is different from reported one:

cli/xsum_sanity_check.c:360:15: runtime error: call to function XXH3_64bits_update through pointer to incorrect function type 'XXH_errorcode (*)(struct XXH3_state_s *, const void *, unsigned long)'
/home/ubuntu/xxhash/./xxhash.h:6339: note: XXH3_64bits_update defined here

Could you describe how you've got this error?

[pengwinsurf]

I get the reported error when I build then run make test-all.

Following that failure if I run make test-all again then I get the second error. So the sequence of commands to get the second error is

make
make test-all
make test-all 

[Cyan4973]

test-all compiles *.o object files using different compilation modes. These object files are for test, and are not generally compatible with the regular binary creation. Mixing object files using different compilation modes doesn't work.

Solution 1 : use make clean after make test-all Solution 2 : don't use test-all. This is a developer target. There is no reason for a user to invoke that target. Prefer make test or even make check.

[pengwinsurf]

Solution 2 doesn't work for me.

I need to ensure I get maximum coverage of xxhash in my tests. When I run make check the coverage is low hence why I tried to use make test-all.

[t-mat]

Please note: We don't recommend the following path for usual users as @Cyan4973 stated. Since there're hidden prerequisites/pitfalls for test-all, we don't recommend it for average users.

(1) Please read our GitHub Actions CI script for ubuntu environment: https://github.com/Cyan4973/xxHash/blob/dev/.github/workflows/ci.yml

This script installs everything to the "standard" Ubuntu VM for GitHub Actions.

(2) It requires gcc-multilib

sudo apt install gcc-multilib

(For contributors) : It's offtopic. But note that in some environment it may be difficult to install gcc-multilib. For example, Alpine Linux due to musl multilib.

(3) It requires lib32gcc-?-dev for specific version of gcc.

(4) Before try something different, always invoke make clean