search

estraier / tkrzw

a set of implementations of DBM
Apache License 2.0
169 stars 21 forks source link

Something wrong with i686 (magic numbers) #8

Closed tieugene closed 3 years ago

tieugene commented 3 years ago

Due this log make check on x86 ends with:

LD_LIBRARY_PATH=.:/usr/local/lib:  ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks \
  --iter 20000 --threads 5 --size 8 --step_unit 4 --max_level 10
Setting: impl=skip num_iterations=20000 value_size=8 num_threads=5
...
.................................................. (00020000)
Synchronizing: ... done (elapsed=0.178743)
Removing done: elapsed_time=0.258109 num_records=0 qps=387433 mem=6052000
  file_size=128 eff_data_size=0 efficiency=0.00%
make[1]: Leaving directory '/builddir/build/BUILD/tkrzw-0.9.3'
RPM build errors:
Get failed: Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
BROKEN_DATA_ERROR: invalid record magic number
make[1]: *** [Makefile:307: check-skipdbm-perf] Error 1
make: *** [Makefile:126: check] Error 2

This is exactly i686 error. Other builds OK.

estraier commented 3 years ago

Thanks for catching it. The first thing I want to distinguish is whether the error happens always or not. I mean, I want to distinguish whether it is a threading problem (race condition) or not. Do you reproduce the same error if you run the check command again?

On Fri, Jan 22, 2021 at 12:56 AM Eugene notifications@github.com wrote:

Due this https://kojipkgs.fedoraproject.org//work/tasks/9580/60149580/build.log log make check on x86 ends with:

LD_LIBRARY_PATH=.:/usr/local/lib: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks \ --iter 20000 --threads 5 --size 8 --step_unit 4 --max_level 10 Setting: impl=skip num_iterations=20000 value_size=8 num_threads=5 ... .................................................. (00020000) Synchronizing: ... done (elapsed=0.178743) Removing done: elapsed_time=0.258109 num_records=0 qps=387433 mem=6052000 file_size=128 eff_data_size=0 efficiency=0.00% make[1]: Leaving directory '/builddir/build/BUILD/tkrzw-0.9.3' RPM build errors: Get failed: Get failed: BROKEN_DATA_ERROR: invalid record magic number Get failed: BROKEN_DATA_ERROR: invalid record magic number Get failed: BROKEN_DATA_ERROR: invalid record magic number Get failed: BROKEN_DATA_ERROR: invalid record magic number BROKEN_DATA_ERROR: invalid record magic number make[1]: [Makefile:307: check-skipdbm-perf] Error 1 make: [Makefile:126: check] Error 2

This is exactly i686 error. All others https://koji.fedoraproject.org/koji/taskinfo?taskID=60149461 builds OK.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/8, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGJVRECTZCGM4M434LIAKTS3BFEFANCNFSM4WNDDYYQ .

tieugene commented 3 years ago

The first thing I want to distinguish is whether the error happens always or not.

The say - not always.

I mean, I want to distinguish whether it is a threading problem (race condition) or not. Do you reproduce the same error if you run the check command again?

As for me I'm at x64 host (Fedora 33) and have no i686 near (Fedora not distributed in i686 version starting from F32). But koji system bulds i686 yet. Somewhere in clouds. Seems anybody with i686 linux host wanted.

musicinmybrain commented 3 years ago

Hi, I am the reviewer in the linked Fedora bug. To clarify, this error occurs every time. I can easily reproduce it outside the Fedora build system, and even without any patches or extra build flags from the distribution. All it takes, on my x86_64 system, is ./configure && make && make check in a 32-bit (i386/i686) chroot.

The tests do occasionally die with “Aborted (core dumped)” in check-treedbm-perf on some platforms, but this is a different problem, and I do not know if it reflects a bug in tkrzw or a problem in the environment. That may be what caused the confusion.

estraier commented 3 years ago

Hi Ben, Thanks for the explanation. I don't have an i686 environment either. By design, Trkzw doesn't depend on specific CPU design, bus width, or endian. So, it's strange to me that the test fails on i686 only. Do you know other environments where the test fails?

On Fri, Jan 22, 2021 at 9:29 PM Ben Beasley notifications@github.com wrote:

Hi, I am the reviewer in the linked Fedora bug. To clarify, this error occurs every time. I can easily reproduce it outside the Fedora build system, and even without any patches or extra build flags from the distribution. All it takes, on my x86_64 system, is ./configure && make && make check in a 32-bit (i386/i686) chroot.

The tests do occasionally die with “Aborted (core dumped)” in check-treedbm-perf on some platforms, but this is a different problem, and I do not know if it reflects a bug in tkrzw or a problem in the environment. That may be what caused the confusion.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/8#issuecomment-765364301, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGJVRGXAD35QWJVTXN4ED3S3FVQZANCNFSM4WNDDYYQ .

tieugene commented 3 years ago

By design, Trkzw doesn't depend on specific CPU design, bus width, or endian. So, it's strange to me that the test fails on i686 only. Do you know other environments where the test fails?

May be the problem is not in exactly i686 but in 32-bit. Fedora's koji system builds for: s390x (IBM System Z), PPC64 (PowerPC), AARCH64 (ARM64), x86_64, i686, ARMv7. But only i686 is 32-bit (don't know about ARMv7) and produces test error.

What about byte align?

musicinmybrain commented 3 years ago

ARMv7 is 32-bit, so it is not just that. And ARM is stricter about alignment than x86, and has a weaker memory model.

Passing --enable-debug or --disable-opt to configure makes no difference, by the way.

I’m happy to try any tests that are helpful.

estraier commented 3 years ago

I'd like to know this test fails or not. The difference is the number of threads is 1.

./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10

If it doesn't fail, please increase the iteration (the value of --iter) to 100000. The point is, I'd like to locate the reproducible point where the segmentation fault (or bus error or anything critical) happens. If it tails, I'd like to see the stack trace.

BTW, running the same test with "valgrind" doesn't report any abnormality on my x86_64 environment. It means that memory violation doesn't happen in the test, on my environment. Is it possible to run the test with valgrind on i686?

musicinmybrain commented 3 years ago 
<mock-chroot> sh-5.1# ./configure --enable-debug && make -j4
#================================================================
# Configuring Tkrzw version 0.9.3 (debug).
#================================================================
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking how to run the C++ preprocessor... g++ -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking whether byte ordering is bigendian... no
checking for main in -lc... yes
checking for main in -lm... yes
checking for main in -lpthread... yes
checking for main in -lrt... yes
checking for main in -lstdc++... yes
checking for main in -ltkrzw... no
checking for stdlib.h... (cached) yes
checking for stdint.h... (cached) yes
checking for unistd.h... (cached) yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking dirent.h usability... yes
checking dirent.h presence... yes
checking for dirent.h... yes
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking utility usability... yes
checking utility presence... yes
checking for utility... yes
checking string usability... yes
checking string presence... yes
checking for string... yes
checking atomic usability... yes
checking atomic presence... yes
checking for atomic... yes
checking mutex usability... yes
checking mutex presence... yes
checking for mutex... yes
checking thread usability... yes
checking thread presence... yes
checking for thread... yes
checking regex usability... yes
checking regex presence... yes
checking for regex... yes
checking cstdint usability... yes
checking cstdint presence... yes
checking for cstdint... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating tkrzw.pc
#================================================================
# Ready to make.
#================================================================
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_lib_common.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_str_util.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_cmd_util.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_thread_util.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_file_util.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_file_mmap.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_file_pos.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_common_impl.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_hash_impl.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_hash.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_tree_impl.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_tree.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_skip_impl.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_skip.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_tiny.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_baby.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_cache.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_std.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_poly.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_shard.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_build_util.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_str_perf.cc
tkrzw_dbm_shard.cc: In constructor ‘tkrzw::ShardDBM::Iterator::Iterator(std::vector<std::shared_ptr<tkrzw::PolyDBM> >*)’:
tkrzw_dbm_shard.cc:373:65: warning: ‘this’ pointer null [-Wnonnull]
  373 |     comp_ = dynamic_cast<const TreeDBM*>(dbm)->GetKeyComparator();
      |                                                                 ^
In file included from tkrzw_dbm_shard.cc:24:
tkrzw_dbm_tree.h:469:17: note: in a call to non-static member function ‘int32_t (* tkrzw::TreeDBM::GetKeyComparator() const)(std::string_view, std::string_view)’
  469 |   KeyComparator GetKeyComparator() const;
      |                 ^~~~~~~~~~~~~~~~
tkrzw_dbm_shard.cc:376:65: warning: ‘this’ pointer null [-Wnonnull]
  376 |     comp_ = dynamic_cast<const BabyDBM*>(dbm)->GetKeyComparator();
      |                                                                 ^
In file included from tkrzw_dbm_shard.cc:15:
tkrzw_dbm_baby.h:329:17: note: in a call to non-static member function ‘int32_t (* tkrzw::BabyDBM::GetKeyComparator() const)(std::string_view, std::string_view)’
  329 |   KeyComparator GetKeyComparator() const;
      |                 ^~~~~~~~~~~~~~~~
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_file_perf.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_perf.cc
g++ -c -I. -I/usr/local/include -I/usr/local/include -DNDEBUG -D_GNU_SOURCE=1 -D_ISOC99_SOURCE=1 -D_POSIX_C_SOURCE=200809L -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D__EXTENSIONS__ -UNDEBUG -D_TKRWX_DEBUG -D_TKRZW_PREFIX="\"/usr/local\"" -D_TKRZW_INCLUDEDIR="\"/usr/local/include\"" -D_TKRZW_LIBDIR="\"/usr/local/lib\"" -D_TKRZW_BINDIR="\"/usr/local/bin\"" -D_TKRZW_LIBEXECDIR="\"/usr/local/libexec\"" -D_TKRZW_APPINC="\"-I/usr/local/include\"" -D_TKRZW_APPLIBS="\"-L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc \"" -D_TKRZW_PKG_VERSION="\"0.9.3\"" -D_TKRZW_LIB_VERSION="\"0.3.0\"" -MMD -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 tkrzw_dbm_util.cc
ar rv libtkrzw.a tkrzw_lib_common.o tkrzw_str_util.o tkrzw_cmd_util.o tkrzw_thread_util.o tkrzw_file_util.o tkrzw_file_mmap.o tkrzw_file_pos.o tkrzw_dbm.o tkrzw_dbm_common_impl.o tkrzw_dbm_hash_impl.o tkrzw_dbm_hash.o tkrzw_dbm_tree_impl.o tkrzw_dbm_tree.o tkrzw_dbm_skip_impl.o tkrzw_dbm_skip.o tkrzw_dbm_tiny.o tkrzw_dbm_baby.o tkrzw_dbm_cache.o tkrzw_dbm_std.o tkrzw_dbm_poly.o tkrzw_dbm_shard.o
ar: creating libtkrzw.a
a - tkrzw_lib_common.o
a - tkrzw_str_util.o
a - tkrzw_cmd_util.o
a - tkrzw_thread_util.o
a - tkrzw_file_util.o
a - tkrzw_file_mmap.o
a - tkrzw_file_pos.o
a - tkrzw_dbm.o
a - tkrzw_dbm_common_impl.o
a - tkrzw_dbm_hash_impl.o
a - tkrzw_dbm_hash.o
a - tkrzw_dbm_tree_impl.o
a - tkrzw_dbm_tree.o
a - tkrzw_dbm_skip_impl.o
a - tkrzw_dbm_skip.o
a - tkrzw_dbm_tiny.o
a - tkrzw_dbm_baby.o
a - tkrzw_dbm_cache.o
a - tkrzw_dbm_std.o
a - tkrzw_dbm_poly.o
a - tkrzw_dbm_shard.o
if uname -a | egrep -i 'SunOS' > /dev/null ; \
  then \
    g++ -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 -shared -Wl,-G,-h,libtkrzw.so.0 -o libtkrzw.so.0.3.0 \
      tkrzw_lib_common.o tkrzw_str_util.o tkrzw_cmd_util.o tkrzw_thread_util.o tkrzw_file_util.o tkrzw_file_mmap.o tkrzw_file_pos.o tkrzw_dbm.o tkrzw_dbm_common_impl.o tkrzw_dbm_hash_impl.o tkrzw_dbm_hash.o tkrzw_dbm_tree_impl.o tkrzw_dbm_tree.o tkrzw_dbm_skip_impl.o tkrzw_dbm_skip.o tkrzw_dbm_tiny.o tkrzw_dbm_baby.o tkrzw_dbm_cache.o tkrzw_dbm_std.o tkrzw_dbm_poly.o tkrzw_dbm_shard.o -L. -L/usr/local/lib -L/usr/local/lib -Wl,-rpath-link,.:/usr/local/lib:.:/usr/local/lib: -Wl,--as-needed -lstdc++ -lrt -lpthread -lm -lc  ; \
  else \
    g++ -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 -shared -Wl,-soname,libtkrzw.so.0 -o libtkrzw.so.0.3.0 \
      tkrzw_lib_common.o tkrzw_str_util.o tkrzw_cmd_util.o tkrzw_thread_util.o tkrzw_file_util.o tkrzw_file_mmap.o tkrzw_file_pos.o tkrzw_dbm.o tkrzw_dbm_common_impl.o tkrzw_dbm_hash_impl.o tkrzw_dbm_hash.o tkrzw_dbm_tree_impl.o tkrzw_dbm_tree.o tkrzw_dbm_skip_impl.o tkrzw_dbm_skip.o tkrzw_dbm_tiny.o tkrzw_dbm_baby.o tkrzw_dbm_cache.o tkrzw_dbm_std.o tkrzw_dbm_poly.o tkrzw_dbm_shard.o -L. -L/usr/local/lib -L/usr/local/lib -Wl,-rpath-link,.:/usr/local/lib:.:/usr/local/lib: -Wl,--as-needed -lstdc++ -lrt -lpthread -lm -lc  ; \
  fi
ln -f -s libtkrzw.so.0.3.0 libtkrzw.so.0
ln -f -s libtkrzw.so.0.3.0 libtkrzw.so
g++ -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 -o tkrzw_build_util tkrzw_build_util.o -L. -L/usr/local/lib -L/usr/local/lib -Wl,-rpath-link,.:/usr/local/lib:.:/usr/local/lib: -Wl,--as-needed -L. -L/usr/local/lib -L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc 
g++ -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 -o tkrzw_str_perf tkrzw_str_perf.o -L. -L/usr/local/lib -L/usr/local/lib -Wl,-rpath-link,.:/usr/local/lib:.:/usr/local/lib: -Wl,--as-needed -L. -L/usr/local/lib -L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc 
g++ -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 -o tkrzw_file_perf tkrzw_file_perf.o -L. -L/usr/local/lib -L/usr/local/lib -Wl,-rpath-link,.:/usr/local/lib:.:/usr/local/lib: -Wl,--as-needed -L. -L/usr/local/lib -L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc 
g++ -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 -o tkrzw_dbm_util tkrzw_dbm_util.o -L. -L/usr/local/lib -L/usr/local/lib -Wl,-rpath-link,.:/usr/local/lib:.:/usr/local/lib: -Wl,--as-needed -L. -L/usr/local/lib -L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc 
g++ -g -O2 -std=c++17 -Wall -Wextra -Wno-unused-parameter -fPIC -fsigned-char -g -O0 -o tkrzw_dbm_perf tkrzw_dbm_perf.o -L. -L/usr/local/lib -L/usr/local/lib -Wl,-rpath-link,.:/usr/local/lib:.:/usr/local/lib: -Wl,--as-needed -L. -L/usr/local/lib -L/usr/local/lib -ltkrzw -lstdc++ -lrt -lpthread -lm -lc 

#================================================================
# Ready to install.
#================================================================
<mock-chroot> sh-5.1# LD_LIBRARY_PATH=$PWD ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10
Setting: impl=skip num_iterations=20000 value_size=8 num_threads=1
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Synchronizing: ... done (elapsed=0.166472)
Setting done: elapsed_time=0.259300 num_records=20000 qps=77131 mem=4684000
  file_size=406816 eff_data_size=320000 efficiency=78.66%

Getting: impl=skip num_iterations=20000 value_size=8 num_threads=1
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Getting done: elapsed_time=0.000716 num_records=20000 qps=27934093 mem=4348000
  file_size=406816 eff_data_size=320000 efficiency=78.66%

Removing: impl=skip num_iterations=20000 value_size=8 num_threads=1
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Synchronizing: ... done (elapsed=0.238865)
Removing done: elapsed_time=0.294029 num_records=0 qps=68021 mem=3996000
  file_size=128 eff_data_size=0 efficiency=0.00%

You can see that it does fail. No segfault or anything; just a BROKEN_DATA_ERROR.

musicinmybrain commented 3 years ago

The same test does not fail under valgrind:

<mock-chroot> sh-5.1# LD_LIBRARY_PATH=$PWD valgrind ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10
==18== Memcheck, a memory error detector
==18== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==18== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==18== Command: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10
==18== 
Setting: impl=skip num_iterations=20000 value_size=8 num_threads=1
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Synchronizing: ... done (elapsed=7.996911)
Setting done: elapsed_time=10.236987 num_records=20000 qps=1954 mem=22440000
  file_size=406816 eff_data_size=320000 efficiency=78.66%

Getting: impl=skip num_iterations=20000 value_size=8 num_threads=1
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Getting done: elapsed_time=23.526716 num_records=20000 qps=850 mem=60320000
  file_size=406816 eff_data_size=320000 efficiency=78.66%

Removing: impl=skip num_iterations=20000 value_size=8 num_threads=1
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Synchronizing: ... done (elapsed=10.655468)
Removing done: elapsed_time=12.734186 num_records=0 qps=1571 mem=60664000
  file_size=128 eff_data_size=0 efficiency=0.00%

==18== 
==18== HEAP SUMMARY:
==18==     in use at exit: 0 bytes in 0 blocks
==18==   total heap usage: 596,566 allocs, 596,566 frees, 34,286,316 bytes allocated
==18== 
==18== All heap blocks were freed -- no leaks are possible
==18== 
==18== For lists of detected and suppressed errors, rerun with: -s
==18== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Nor does it fail under valgrind with --iter 100000, or --threads 5.

estraier commented 3 years ago

Thanks. If the test with --threads 1 fails, the problem is not related to threading. There should be some deterministic wrong behavior in Tkrzw's implementation. But, one weird thing is that running on valgrind avoids the failure. Anyway, I'll go over the code to locate the cause.

On Sat, Jan 23, 2021 at 2:39 AM Ben Beasley notifications@github.com wrote:

The same test does not fail under valgrind:

sh-5.1# LD_LIBRARY_PATH=$PWD valgrind ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10 ==18== Memcheck, a memory error detector ==18== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==18== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info ==18== Command: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10 ==18== Setting: impl=skip num_iterations=20000 value_size=8 num_threads=1 .................................................. (00001000) .................................................. (00002000) .................................................. (00003000) .................................................. (00004000) .................................................. (00005000) .................................................. (00006000) .................................................. (00007000) .................................................. (00008000) .................................................. (00009000) .................................................. (00010000) .................................................. (00011000) .................................................. (00012000) .................................................. (00013000) .................................................. (00014000) .................................................. (00015000) .................................................. (00016000) .................................................. (00017000) .................................................. (00018000) .................................................. (00019000) .................................................. (00020000) Synchronizing: ... done (elapsed=7.996911) Setting done: elapsed_time=10.236987 num_records=20000 qps=1954 mem=22440000 file_size=406816 eff_data_size=320000 efficiency=78.66% Getting: impl=skip num_iterations=20000 value_size=8 num_threads=1 .................................................. (00001000) .................................................. (00002000) .................................................. (00003000) .................................................. (00004000) .................................................. (00005000) .................................................. (00006000) .................................................. (00007000) .................................................. (00008000) .................................................. (00009000) .................................................. (00010000) .................................................. (00011000) .................................................. (00012000) .................................................. (00013000) .................................................. (00014000) .................................................. (00015000) .................................................. (00016000) .................................................. (00017000) .................................................. (00018000) .................................................. (00019000) .................................................. (00020000) Getting done: elapsed_time=23.526716 num_records=20000 qps=850 mem=60320000 file_size=406816 eff_data_size=320000 efficiency=78.66% Removing: impl=skip num_iterations=20000 value_size=8 num_threads=1 .................................................. (00001000) .................................................. (00002000) .................................................. (00003000) .................................................. (00004000) .................................................. (00005000) .................................................. (00006000) .................................................. (00007000) .................................................. (00008000) .................................................. (00009000) .................................................. (00010000) .................................................. (00011000) .................................................. (00012000) .................................................. (00013000) .................................................. (00014000) .................................................. (00015000) .................................................. (00016000) .................................................. (00017000) .................................................. (00018000) .................................................. (00019000) .................................................. (00020000) Synchronizing: ... done (elapsed=10.655468) Removing done: elapsed_time=12.734186 num_records=0 qps=1571 mem=60664000 file_size=128 eff_data_size=0 efficiency=0.00% ==18== ==18== HEAP SUMMARY: ==18== in use at exit: 0 bytes in 0 blocks ==18== total heap usage: 596,566 allocs, 596,566 frees, 34,286,316 bytes allocated ==18== ==18== All heap blocks were freed -- no leaks are possible ==18== ==18== For lists of detected and suppressed errors, rerun with: -s ==18== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Nor does it fail under valgrind with --iter 100000, or --threads 5. — You are receiving this because you commented. Reply to this email directly, view it on GitHub , or unsubscribe .
estraier commented 3 years ago

I'm now trying to verify this issue on VMware. Where's the ISO image of the exact version on which you found the bug?

musicinmybrain commented 3 years ago

Fedora Linux builds some “multilib” i686 packages as part of the x86_64 architecture, but doesn’t build a complete 32-bit release anymore.

I expect you can best reproduce this by using the x86_64 DVD ISO for the just-released Fedora 34 from here, and adding -m32 to the compiler and linker flags. I’ll try it out myself and report back.

musicinmybrain commented 3 years ago

OK, so I started the live DVD I linked, started “Terminal” (gnome-terminal), and ran:

sudo dnf install gcc-c++ libstdc++-devel.x86_64 libstdc++-devel.i686 libgcc.i686 glibc-devel.i686

wget https://github.com/estraier/tkrzw/archive/0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip
unzip 0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip
cd tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305/

CXXFLAGS=-m32 LDFLAGS=-m32 ./configure
# Change the number of parallel jobs as desired; my VM had 4 CPUs and adequate memory
make -j4
make check

After a while (my workstation is an antique) I get:

make check-skipdbm-perf
make[1]: Entering directory '/home/liveuser/tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305'
rm -Rf casket*
LD_LIBRARY_PATH=.:/usr/local/lib:  ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks \
  --iter 20000 --threads 5 --size 8 --step_unit 4 --max_level 10
Setting: impl=skip num_iterations=20000 value_size=8 num_threads=5
.................................................. (00001000)
.................................................. (00002000)
.................................................. (00003000)
.................................................. (00004000)
.................................................. (00005000)
.................................................. (00006000)
.................................................. (00007000)
.................................................. (00008000)
.................................................. (00009000)
.................................................. (00010000)
.................................................. (00011000)
.................................................. (00012000)
.................................................. (00013000)
.................................................. (00014000)
.................................................. (00015000)
.................................................. (00016000)
.................................................. (00017000)
.................................................. (00018000)
.................................................. (00019000)
.................................................. (00020000)
Synchronizing: ... done (elapsed=1.172894)
Setting done: elapsed_time=2.466667 num_records=100000 qps=40541 mem=8828000
  file_size=2033480 eff_data_size=1600000 efficiency=78.68%

Getting: impl=skip num_iterations=20000 value_size=8 num_threads=5
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic number
Get failed: BROKEN_DATA_ERROR: invalid record magic numberGet failed: BROKEN_DATA_ERROR: invalid record magic number

Getting done: elapsed_time=0.020106 num_records=100000 qps=4973680 mem=7232000
  file_size=2033480 eff_data_size=1600000 efficiency=78.68%

Manually running the test we were looking at previously,

LD_LIBRARY_PATH=$PWD ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks --iter 20000 --threads 1 --size 8 --step_unit 4 --max_level 10

fails too.

So it seems that’s a good way to reproduce it.

estraier commented 3 years ago

Thanks. Reproduction is the greatest step to fix an issue!

BTW, the ISO image you wrote is for 64 bit. Is it intended?

On Thu, Apr 29, 2021 at 5:03 AM Ben Beasley @.***> wrote:

OK, so I started the live DVD I linked, started “Terminal” (gnome-terminal), and ran:

sudo dnf install gcc-c++ libstdc++-devel.x86_64 libstdc++-devel.i686 libgcc.i686 glibc-devel.i686

wget https://github.com/estraier/tkrzw/archive/0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip

unzip 0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip cd tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305/

CXXFLAGS=-m32 LDFLAGS=-m32 ./configure

Change the number of parallel jobs as desired; my VM had 4 CPUs and adequate memory

make -j4

make check

After a while (my workstation is an antique) I get:

make check-skipdbm-perf

make[1]: Entering directory '/home/liveuser/tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305'

rm -Rf casket*

LD_LIBRARY_PATH=.:/usr/local/lib: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks \

--iter 20000 --threads 5 --size 8 --step_unit 4 --max_level 10

Setting: impl=skip num_iterations=20000 value_size=8 num_threads=5

.................................................. (00001000)

.................................................. (00002000)

.................................................. (00003000)

.................................................. (00004000)

.................................................. (00005000)

.................................................. (00006000)

.................................................. (00007000)

.................................................. (00008000)

.................................................. (00009000)

.................................................. (00010000)

.................................................. (00011000)

.................................................. (00012000)

.................................................. (00013000)

.................................................. (00014000)

.................................................. (00015000)

.................................................. (00016000)

.................................................. (00017000)

.................................................. (00018000)

.................................................. (00019000)

.................................................. (00020000)

Synchronizing: ... done (elapsed=1.172894)

Setting done: elapsed_time=2.466667 num_records=100000 qps=40541 mem=8828000

file_size=2033480 eff_data_size=1600000 efficiency=78.68%

Getting: impl=skip num_iterations=20000 value_size=8 num_threads=5

Get failed: BROKEN_DATA_ERROR: invalid record magic number

Get failed: BROKEN_DATA_ERROR: invalid record magic number

Get failed: BROKEN_DATA_ERROR: invalid record magic number

Get failed: BROKEN_DATA_ERROR: invalid record magic numberGet failed: BROKEN_DATA_ERROR: invalid record magic number

Getting done: elapsed_time=0.020106 num_records=100000 qps=4973680 mem=7232000

file_size=2033480 eff_data_size=1600000 efficiency=78.68%

So it seems that’s a good way to reproduce it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/8#issuecomment-828740461, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGJVRE2UZZLPWYEL6Z3SW3TLBSYXANCNFSM4WNDDYYQ .

estraier commented 3 years ago

Ah, now I understand. Giving "-m32" is the key.

On Thu, Apr 29, 2021 at 11:47 AM Mikio Hirabayashi @.***> wrote:

Thanks. Reproduction is the greatest step to fix an issue!

BTW, the ISO image you wrote is for 64 bit. Is it intended?

On Thu, Apr 29, 2021 at 5:03 AM Ben Beasley @.***> wrote:

OK, so I started the live DVD I linked, started “Terminal” ( gnome-terminal), and ran:

sudo dnf install gcc-c++ libstdc++-devel.x86_64 libstdc++-devel.i686 libgcc.i686 glibc-devel.i686

wget https://github.com/estraier/tkrzw/archive/0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip

unzip 0d7badb65abcac72381ecc6fbd8f980ce6a53305.zip cd tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305/

CXXFLAGS=-m32 LDFLAGS=-m32 ./configure

Change the number of parallel jobs as desired; my VM had 4 CPUs and adequate memory

make -j4

make check

After a while (my workstation is an antique) I get:

make check-skipdbm-perf

make[1]: Entering directory '/home/liveuser/tkrzw-0d7badb65abcac72381ecc6fbd8f980ce6a53305'

rm -Rf casket*

LD_LIBRARY_PATH=.:/usr/local/lib: ./tkrzw_dbm_perf sequence --dbm skip --file mmap-para --path casket.tks \

--iter 20000 --threads 5 --size 8 --step_unit 4 --max_level 10

Setting: impl=skip num_iterations=20000 value_size=8 num_threads=5

.................................................. (00001000)

.................................................. (00002000)

.................................................. (00003000)

.................................................. (00004000)

.................................................. (00005000)

.................................................. (00006000)

.................................................. (00007000)

.................................................. (00008000)

.................................................. (00009000)

.................................................. (00010000)

.................................................. (00011000)

.................................................. (00012000)

.................................................. (00013000)

.................................................. (00014000)

.................................................. (00015000)

.................................................. (00016000)

.................................................. (00017000)

.................................................. (00018000)

.................................................. (00019000)

.................................................. (00020000)

Synchronizing: ... done (elapsed=1.172894)

Setting done: elapsed_time=2.466667 num_records=100000 qps=40541 mem=8828000

file_size=2033480 eff_data_size=1600000 efficiency=78.68%

Getting: impl=skip num_iterations=20000 value_size=8 num_threads=5

Get failed: BROKEN_DATA_ERROR: invalid record magic number

Get failed: BROKEN_DATA_ERROR: invalid record magic number

Get failed: BROKEN_DATA_ERROR: invalid record magic number

Get failed: BROKEN_DATA_ERROR: invalid record magic numberGet failed: BROKEN_DATA_ERROR: invalid record magic number

Getting done: elapsed_time=0.020106 num_records=100000 qps=4973680 mem=7232000

file_size=2033480 eff_data_size=1600000 efficiency=78.68%

So it seems that’s a good way to reproduce it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/estraier/tkrzw/issues/8#issuecomment-828740461, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQGJVRE2UZZLPWYEL6Z3SW3TLBSYXANCNFSM4WNDDYYQ .

estraier commented 3 years ago

This issue could be reproducible on Ubuntu 20.01 x64 if -m32 is attached to the build command. And, I found the bug. A pointer variable could overflow on 32-bit systems. I committed the CL to fix it.

Thanks so much. I appreciate people who test the product on various environments/settings.

tieugene commented 3 years ago

Fedora 32..35, RH8 - ok

tieugene commented 3 years ago

Seems nobody else will report this bug