estraier / tkrzw

a set of implementations of DBM
Apache License 2.0
168 stars 20 forks source link

Most implementations have low tolerance to corrupted database files #46

Open debrouxl opened 5 months ago

debrouxl commented 5 months ago

Hello,

tkrzw_crashes_202404_01.tar.gz

Here's a tarball containing a set of redundant corrupted files which crash at least one of the commands listed below, and the corresponding terminal output (crashes_*.txt files):

tkrzw_dbm_util inspect "$file"
tkrzw_dbm_util rebuild --restore "$file"
tkrzw_dbm_util list --items 4294967296 "$file"
tkrzw_dbm_util set "$file" key0 value0
tkrzw_dbm_util remove "$file" key0

in tkrzw 1.0.27 built thusly under Debian sid amd64:

apt download tkrzw
# As of 2024/04, on Debian sid amd64, this downloads 1.0.27, which is the latest upstream version.
cd tkrzw-...
AFL_USE_ASAN=1 CC=afl-clang-fast CXX=afl-clang-fast++ dpkg-buildpackage -us -uc -b -j4
dpkg -i ... # currently libtkrzw1t64_1.0.27-2_amd64.deb tkrzw-utils_1.0.27-2_amd64.deb

Most of the crashes on those files are controlled asserts caused by attempts to allocate terabytes of memory or more; however, there are also wild pointer accesses, heap-based buffer overflows, etc. Only the tkh file type didn't fall to afl-fuzz (yet); tkmb tkmc tkmt tks tksh tkst tkt did, most of them for all five commands, often within the first few seconds of fuzzing, if not the first dozens of milliseconds. Some of them reached ~2% crash rate.

FTR:

The initial corpus of valid files was built by

mkdir /dev/shm/tkrzw_fuzz
cd /dev/shm/tkrzw_fuzz
for ext in tkh tkt tks tkmt tkmb tkmc tksh tkst; do
    mkdir "input_$ext"
    tkrzw_dbm_util create --alloc_init 4096 --buckets 1024 "input_$ext/empty.$ext"
    tkrzw_dbm_util create --alloc_init 4096 --buckets 1024 "input_$ext/one.$ext"
    tkrzw_dbm_util set "input_$ext/one.$ext" key1 value1
done

for ext in tkh tkt; do
    for crc in 8 16 32; do
        for comp in zlib zstd lz4 lzma rc4 aes; do
            cp "input_$ext/one.$ext" "input_$ext/one_crc${crc}_comp${comp}.$ext"
            tkrzw_dbm_util rebuild --record_crc "$crc" --record_comp "$comp" "input_$ext/one_crc${crc}_comp${comp}.$ext"
        done
    done
done

tkrzw_dbm_util create --alloc_init 4096 --buckets 1024 input_tkh/ten.tkh
tkrzw_dbm_util create --alloc_init 4096 --buckets 1024 input_tkt/ten.tkt
for i in `seq 0 9`; do
    tkrzw_dbm_util set input_tkh/ten.tkh "key$i" "value$i"
    tkrzw_dbm_util set input_tkt/ten.tkt "key$i" "value$i"
done

afl-fuzz invocations:

mkdir /dev/shm/tkrzw_fuzz/tk_tmpdir{m,1,2,3,4,5,6,7}_{wd,afl,tmpdir}

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkh -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkt -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tks -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkmt -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkmb -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkmc -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tksh -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkst -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util inspect @@

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util rebuild --restore @@

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util list --items 4294967296 @@

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util set @@ key0 value0

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util remove @@ key0

I then massaged the output folders for easier use on the maintainer side :)

debrouxl commented 4 months ago

Ping ? It's been a month :)

estraier commented 4 months ago

Thanks for the report (and sorry for the belated response).

Let me know of the goal of this experiment. If the process crashes during an operation setting record "A", it's natural that the record "A" is not recovered. If the process crashes during an operation setting record "B" after setting record "A", it is expected that record "A" is recovered. And, your code seems to just setting one record and the process crashes during the operation. Then, what's the expected behavior?

debrouxl commented 4 months ago

The expected behaviour is not to crash, at the very least :) While it's clear that the attempted operations, especially adding / removing / modifying records, do not have to complete successfully on a broken database, the database system must not allocate excessive amounts of memory (or abort the program through an assert trying to do that), read memory outside the allocated areas (Out Of Bound reads from stack, heap, global variables), write memory outside the allocated areas (Out Of Bounds writes, a.k.a. "memory corruption"), divide by zero, perform improper memory accesses which yield bus errors, etc. Beyond being user-unfriendly, especially a complete failure to recover a database which was already broken and usually contains data useful to the user (instead of a failure to salvage a subset of the data, when possible), the aforementioned classes of misbehaviour are vulnerabilities. Some of them (and probably all of the ones the fuzzer found in tkrzw so far) are just Denial of Service (DoS), but memory corruption often yields arbitrary code execution / remote code execution, if the corruption primitive is powerful enough and/or can be sufficiently repeated.

For the record: