Double-free in RNG_test

[icefoxen]

Hi, I was playing around with this and couldn't reproduce the results. Minor issue was that runtests.sh was missing the #!/bin/bash at the beginning, so non-standard shells chokoe on it... but the bigger issue was that often tests aborted with a double-free, while being reported as successful. Example:

> ./runtests.sh                            [11:50:31]
Testing 512GB  of data per run
Note: running the tests longer could expose new failures.
# RUNNING testmitchellmoore Outputting result to  testmitchellmoore.log
Failure!
# RUNNING testmersennetwister Outputting result to  testmersennetwister.log
free(): double free detected in tcache 2
./runtests.sh: line 11: 10915 Broken pipe             ./$t
     10916 Aborted                 | ./RNG_test stdin64 -tlmax $MEM > $filelog
Success!
# RUNNING testxorshift-k4 Outputting result to  testxorshift-k4.log
Failure!
# RUNNING testxorshift-k5 Outputting result to  testxorshift-k5.log
Failure!
# RUNNING testwidynski Outputting result to  testwidynski.log
free(): double free detected in tcache 2
./runtests.sh: line 11: 10935 Broken pipe             ./$t
     10936 Aborted                 | ./RNG_test stdin64 -tlmax $MEM > $filelog
Success!
# RUNNING testaesctr Outputting result to  testaesctr.log
free(): double free detected in tcache 2
./runtests.sh: line 11: 10941 Broken pipe             ./$t
     10942 Aborted                 | ./RNG_test stdin64 -tlmax $MEM > $filelog
Success!
# RUNNING testaesdragontamer Outputting result to  testaesdragontamer.log
free(): double free detected in tcache 2
./runtests.sh: line 11: 10947 Broken pipe             ./$t
     10948 Aborted                 | ./RNG_test stdin64 -tlmax $MEM > $filelog
Success!
# RUNNING testv8xorshift128plus -H Outputting result to  testv8xorshift128plus-H.log
free(): double free detected in tcache 2
./runtests.sh: line 11: 10953 Broken pipe             ./$t
     10954 Aborted                 | ./RNG_test stdin64 -tlmax $MEM > $filelog
Success!
# RUNNING testv8xorshift128plus Outputting result to  testv8xorshift128plus.log
Failure!
# RUNNING testxorshift128plus -H Outputting result to  testxorshift128plus-H.log
free(): double free detected in tcache 2
./runtests.sh: line 11: 10965 Broken pipe             ./$t
     10966 Aborted                 | ./RNG_test stdin64 -tlmax $MEM > $filelog
Success!
# RUNNING testxorshift128plus Outputting result to  testxorshift128plus.log
...

I tracked the double-free down enough to this backtrace:

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f996e51d537 in __GI_abort () at abort.c:79
#2  0x00007f996e5766c8 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f996e684e31 "%s\n")
    at ../sysdeps/posix/libc_fatal.c:155
#3  0x00007f996e57d9ba in malloc_printerr (str=str@entry=0x7f996e687150 "free(): double free detected in tcache 2")
    at malloc.c:5347
#4  0x00007f996e57efb5 in _int_free (av=0x7f996e6b6b80 <main_arena>, p=0x56554fb6cad0, have_lock=0) at malloc.c:4201
#5  0x000056554dd8bd76 in std::vector<PractRand::TestResult, std::allocator<PractRand::TestResult> >::~vector() ()
#6  0x000056554dd76595 in show_checkpoint(TestManager*, int, unsigned long long, double, bool, double, bool) [clone .cold] ()
#7  0x000056554dd7c609 in main ()

but unfortunately looking at show_checkpoint() didn't show anything obviously wrong and I don't have time to keep digging into this problem.

Thank you for your work!

Platform: Debian 11 Linux, x86_64, gcc

[lemire]

Thanks for the report. Note that the instructions specifically say "Go to a bash shell".

[lemire]

Ok. I looked at this and RNG_test is built with the following command...

    (unzip -qo PractRand_0.93.zip -d build && cd build &&  cat ../practrand-0.93-bigbuffer.patch | patch -p0 && $(CXX) -std=c++14 -c src/*.cpp src/RNGs/*.cpp src/RNGs/other/*.cpp -O3 -Iinclude -pthread &&  ar rcs libPractRand.a *.o &&  $(CXX) -std=c++14 -o RNG_test tools/RNG_test.cpp libPractRand.a -O3 -Iinclude -pthread && cp RNG_test ..

It runs without issue for me, but if you are encountering memory corruption in RNG_test, then you should report the issue to the PractRand project.

I will add bash markers to the script, but I cannot fix memory corruption in RNG_test.

[lemire]

To be clear, I did not write a single line of code that gets compiled into RNG_test. It is entirely from the PractRand project.

[icefoxen]

Thank you! I'll do some more testing and forward it on to PractRand.