cowsql / raft

Asynchronous C implementation of the Raft consensus protocol
https://raft.readthedocs.io
Other
53 stars 6 forks source link

0.18.3: test suite fails #125

Closed kloczek closed 10 months ago

kloczek commented 10 months ago

looks like something is wrong and test suite fails in few units.

kloczek commented 10 months ago
```console + cd raft-0.18.3 + /usr/bin/make -O -j48 V=1 VERBOSE=1 check /usr/bin/make test/unit/core test/integration/core test/fuzzy/core test/unit/uv test/integration/uv libtest.la make[1]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' /usr/bin/gcc -DPACKAGE_NAME=\"raft\" -DPACKAGE_TARNAME=\"raft\" -DPACKAGE_VERSION=\"0.18.3\" -DPACKAGE_STRING=\"raft\ 0.18.3\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"raft\" -DVERSION=\"0.18.3\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DHAVE_WCHAR_H=1 -DSTDC_HEADERS=1 -D_ALL_SOURCE=1 -D_DARWIN_C_SOURCE=1 -D_GNU_SOURCE=1 -D_HPUX_ALT_XOPEN_SOCKET_API=1 -D_NETBSD_SOURCE=1 -D_OPENBSD_SOURCE=1 -D_POSIX_PTHREAD_SEMANTICS=1 -D__STDC_WANT_IEC_60559_ATTRIBS_EXT__=1 -D__STDC_WANT_IEC_60559_BFP_EXT__=1 -D__STDC_WANT_IEC_60559_DFP_EXT__=1 -D__STDC_WANT_IEC_60559_EXT__=1 -D__STDC_WANT_IEC_60559_FUNCS_EXT__=1 -D__STDC_WANT_IEC_60559_TYPES_EXT__=1 -D__STDC_WANT_LIB_EXT2__=1 -D__STDC_WANT_MATH_SPEC_FUNCS__=1 -D_TANDEM_SOURCE=1 -D__EXTENSIONS__=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_STDIO_H=1 -DHAVE_ASSERT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_LINUX_IO_URING_H=1 -DHAVE_LINUX_AIO_ABI_H=1 -DHAVE_DECL_RWF_NOWAIT=1 -I. -DLZ4_AVAILABLE -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -c -o src/test_unit_core-random.o `test -f 'src/random.c' || echo './'`src/random.c make[1]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' make[1]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' [..] make[1]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' make[1]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' /bin/sh ./libtool --tag=CC --mode=link /usr/bin/gcc -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -no-install -Wl,--gc-sections -Wl,--as-needed -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--build-id=sha1 -o test/integration/core test/integration/main_core.o test/integration/test_apply.o test/integration/test_assign.o test/integration/test_barrier.o test/integration/test_bootstrap.o test/integration/test_digest.o test/integration/test_election.o test/integration/test_fixture.o test/integration/test_heap.o test/integration/test_init.o test/integration/test_membership.o test/integration/test_recover.o test/integration/test_replication.o test/integration/test_snapshot.o test/integration/test_start.o test/integration/test_strerror.o test/integration/test_tick.o test/integration/test_transfer.o libtest.la libraft.la libtool: link: /usr/bin/gcc -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -Wl,--gc-sections -Wl,--as-needed -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--build-id=sha1 -o test/integration/core test/integration/main_core.o test/integration/test_apply.o test/integration/test_assign.o test/integration/test_barrier.o test/integration/test_bootstrap.o test/integration/test_digest.o test/integration/test_election.o test/integration/test_fixture.o test/integration/test_heap.o test/integration/test_init.o test/integration/test_membership.o test/integration/test_recover.o test/integration/test_replication.o test/integration/test_snapshot.o test/integration/test_start.o test/integration/test_strerror.o test/integration/test_tick.o test/integration/test_transfer.o ./.libs/libtest.a ./.libs/libraft.so -llz4 -luv -Wl,-rpath -Wl,/home/tkloczko/rpmbuild/BUILD/raft-0.18.3/.libs make[1]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' make[1]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' /bin/sh ./libtool --tag=CC --mode=link /usr/bin/gcc -DLZ4_AVAILABLE -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -no-install -llz4 -Wl,--gc-sections -Wl,--as-needed -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--build-id=sha1 -o test/integration/uv src/test_integration_uv-byte.o src/test_integration_uv-client.o src/test_integration_uv-compress.o src/test_integration_uv-configuration.o src/test_integration_uv-convert.o src/test_integration_uv-election.o src/test_integration_uv-entry.o src/test_integration_uv-err.o src/test_integration_uv-flags.o src/test_integration_uv-heap.o src/test_integration_uv-legacy.o src/test_integration_uv-lifecycle.o src/test_integration_uv-log.o src/test_integration_uv-membership.o src/test_integration_uv-progress.o src/test_integration_uv-random.o src/test_integration_uv-raft.o src/test_integration_uv-recv.o src/test_integration_uv-recv_append_entries.o src/test_integration_uv-recv_append_entries_result.o src/test_integration_uv-recv_request_vote.o src/test_integration_uv-recv_request_vote_result.o src/test_integration_uv-recv_install_snapshot.o src/test_integration_uv-recv_timeout_now.o src/test_integration_uv-restore.o src/test_integration_uv-replication.o src/test_integration_uv-snapshot.o src/test_integration_uv-state.o src/test_integration_uv-syscall.o src/test_integration_uv-task.o src/test_integration_uv-tick.o src/test_integration_uv-tracing.o src/test_integration_uv-fixture.o src/test_integration_uv-uv.o src/test_integration_uv-uv_append.o src/test_integration_uv-uv_encoding.o src/test_integration_uv-uv_finalize.o src/test_integration_uv-uv_fs.o src/test_integration_uv-uv_ip.o src/test_integration_uv-uv_list.o src/test_integration_uv-uv_metadata.o src/test_integration_uv-uv_os.o src/test_integration_uv-uv_prepare.o src/test_integration_uv-uv_recv.o src/test_integration_uv-uv_segment.o src/test_integration_uv-uv_send.o src/test_integration_uv-uv_snapshot.o src/test_integration_uv-uv_tcp.o src/test_integration_uv-uv_tcp_listen.o src/test_integration_uv-uv_tcp_connect.o src/test_integration_uv-uv_truncate.o src/test_integration_uv-uv_writer.o test/integration/test_integration_uv-main_uv.o test/integration/test_integration_uv-test_uv_init.o test/integration/test_integration_uv-test_uv_append.o test/integration/test_integration_uv-test_uv_bootstrap.o test/integration/test_integration_uv-test_uv_load.o test/integration/test_integration_uv-test_uv_recover.o test/integration/test_integration_uv-test_uv_recv.o test/integration/test_integration_uv-test_uv_send.o test/integration/test_integration_uv-test_uv_set_term.o test/integration/test_integration_uv-test_uv_tcp_connect.o test/integration/test_integration_uv-test_uv_tcp_listen.o test/integration/test_integration_uv-test_uv_snapshot_put.o test/integration/test_integration_uv-test_uv_truncate.o test/integration/test_integration_uv-test_uv_truncate_snapshot.o libtest.la -luv libtool: link: /usr/bin/gcc -DLZ4_AVAILABLE -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -Wl,--gc-sections -Wl,--as-needed -flto=auto -flto-partition=none -fuse-linker-plugin -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,--build-id=sha1 -o test/integration/uv src/test_integration_uv-byte.o src/test_integration_uv-client.o src/test_integration_uv-compress.o src/test_integration_uv-configuration.o src/test_integration_uv-convert.o src/test_integration_uv-election.o src/test_integration_uv-entry.o src/test_integration_uv-err.o src/test_integration_uv-flags.o src/test_integration_uv-heap.o src/test_integration_uv-legacy.o src/test_integration_uv-lifecycle.o src/test_integration_uv-log.o src/test_integration_uv-membership.o src/test_integration_uv-progress.o src/test_integration_uv-random.o src/test_integration_uv-raft.o src/test_integration_uv-recv.o src/test_integration_uv-recv_append_entries.o src/test_integration_uv-recv_append_entries_result.o src/test_integration_uv-recv_request_vote.o src/test_integration_uv-recv_request_vote_result.o src/test_integration_uv-recv_install_snapshot.o src/test_integration_uv-recv_timeout_now.o src/test_integration_uv-restore.o src/test_integration_uv-replication.o src/test_integration_uv-snapshot.o src/test_integration_uv-state.o src/test_integration_uv-syscall.o src/test_integration_uv-task.o src/test_integration_uv-tick.o src/test_integration_uv-tracing.o src/test_integration_uv-fixture.o src/test_integration_uv-uv.o src/test_integration_uv-uv_append.o src/test_integration_uv-uv_encoding.o src/test_integration_uv-uv_finalize.o src/test_integration_uv-uv_fs.o src/test_integration_uv-uv_ip.o src/test_integration_uv-uv_list.o src/test_integration_uv-uv_metadata.o src/test_integration_uv-uv_os.o src/test_integration_uv-uv_prepare.o src/test_integration_uv-uv_recv.o src/test_integration_uv-uv_segment.o src/test_integration_uv-uv_send.o src/test_integration_uv-uv_snapshot.o src/test_integration_uv-uv_tcp.o src/test_integration_uv-uv_tcp_listen.o src/test_integration_uv-uv_tcp_connect.o src/test_integration_uv-uv_truncate.o src/test_integration_uv-uv_writer.o test/integration/test_integration_uv-main_uv.o test/integration/test_integration_uv-test_uv_init.o test/integration/test_integration_uv-test_uv_append.o test/integration/test_integration_uv-test_uv_bootstrap.o test/integration/test_integration_uv-test_uv_load.o test/integration/test_integration_uv-test_uv_recover.o test/integration/test_integration_uv-test_uv_recv.o test/integration/test_integration_uv-test_uv_send.o test/integration/test_integration_uv-test_uv_set_term.o test/integration/test_integration_uv-test_uv_tcp_connect.o test/integration/test_integration_uv-test_uv_tcp_listen.o test/integration/test_integration_uv-test_uv_snapshot_put.o test/integration/test_integration_uv-test_uv_truncate.o test/integration/test_integration_uv-test_uv_truncate_snapshot.o -llz4 ./.libs/libtest.a -luv make[1]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' /usr/bin/make check-TESTS make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' PASS: test/unit/uv make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' FAIL: test/unit/core make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' FAIL: test/fuzzy/core make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' FAIL: test/integration/core make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' FAIL: test/integration/uv make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' make[2]: Entering directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3' =================================== raft 0.18.3: ./test-suite.log =================================== # TOTAL: 5 # PASS: 1 # SKIP: 0 # XFAIL: 0 # FAIL: 4 # XPASS: 0 # ERROR: 0 .. contents:: :depth: 2 FAIL: test/unit/core ==================== Running test suite with seed 0x4f361f11... RandomWithinRange/seed [ OK ] [ 0.00000775 / 0.00000672 CPU ] RandomWithinRange/sequence [ OK ] [ 0.00000809 / 0.00000678 CPU ] RandomWithinRange/first [ OK ] [ 0.00000343 / 0.00000276 CPU ] QUEUE_FOREACH/two [ OK ] [ 0.00000382 / 0.00000299 CPU ] QUEUE_FOREACH/one [ OK ] [ 0.00000277 / 0.00000236 CPU ] QUEUE_FOREACH/zero [ OK ] [ 0.00000320 / 0.00000251 CPU ] QUEUE_TAIL/three [ OK ] [ 0.00000786 / 0.00000660 CPU ] QUEUE_TAIL/two [ OK ] [ 0.00000315 / 0.00000245 CPU ] QUEUE_TAIL/one [ OK ] [ 0.00000350 / 0.00000281 CPU ] QUEUE_REMOVE/success [ OK ] [ 0.00000187 / 0.00000166 CPU ] QUEUE_REMOVE/second [ OK ] [ 0.00000240 / 0.00000219 CPU ] QUEUE_REMOVE/first [ OK ] [ 0.00000295 / 0.00000271 CPU ] QUEUE_PUSH/two [ OK ] [ 0.00000295 / 0.00000224 CPU ] QUEUE_PUSH/one [ OK ] [ 0.00000326 / 0.00000258 CPU ] QUEUE_IS_EMPTY/no [ OK ] [ 0.00000280 / 0.00000226 CPU ] QUEUE_IS_EMPTY/yes [ OK ] [ 0.00000236 / 0.00000212 CPU ] logRestore/wipe [ OK ] [ 0.00003947 / 0.00003897 CPU ] logRestore/initial [ OK ] [ 0.00000587 / 0.00000570 CPU ] logSnapshot/wrap [ OK ] [ 0.00003431 / 0.00003388 CPU ] logSnapshot/lessThanHighestIndex [ OK ] [ 0.00002821 / 0.00002807 CPU ] logSnapshot/trailingMatchesOutstanding [ OK ] [ 0.00003271 / 0.00003217 CPU ] logSnapshot/trailingHigherThanNumEntries [ OK ] [ 0.00002852 / 0.00002831 CPU ] logSnapshot/trailing [ OK ] [ 0.00002750 / 0.00002738 CPU ] logTruncate/acquiredOom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00002214 / 0.00002189 CPU ] logTruncate/acquireAppend [ OK ] [ 0.00010701 / 0.00010694 CPU ] logTruncate/acquired [ OK ] [ 0.00002901 / 0.00002876 CPU ] logTruncate/batch [ OK ] [ 0.00002680 / 0.00002668 CPU ] logTruncate/referenced [ OK ] [ 0.00003697 / 0.00003686 CPU ] logTruncate/wrap [ OK ] [ 0.00003720 / 0.00003679 CPU ] logTruncate/lastOfTwo [ OK ] [ 0.00003312 / 0.00003307 CPU ] logTruncate/lastOfOne [ OK ] [ 0.00003014 / 0.00002984 CPU ] logAcquire/oom [ OK ] [ 0.00002924 / 0.00002886 CPU ] logAcquire/outOfRange [ OK ] [ 0.00003109 / 0.00003076 CPU ] logAcquire/batch [ OK ] [ 0.00003377 / 0.00003339 CPU ] logAcquire/wrap [ OK ] [ 0.00003049 / 0.00003028 CPU ] logAcquire/two [ OK ] [ 0.00002882 / 0.00002874 CPU ] logAcquire/one [ OK ] [ 0.00002933 / 0.00002920 CPU ] logAppendConfiguration/oom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00001178 / 0.00001142 CPU ] heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00001055 / 0.00001042 CPU ] logAppend/busy [ OK ] [ 0.00003232 / 0.00003195 CPU ] logAppend/oomRefs [ OK ] [ 0.00010925 / 0.00010386 CPU ] logAppend/oom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00000263 / 0.00000235 CPU ] heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00001402 / 0.00001358 CPU ] logAppend/batch [ OK ] [ 0.00003043 / 0.00002991 CPU ] logAppend/wrap [ OK ] [ 0.00003565 / 0.00003503 CPU ] logAppend/many [ OK ] [ 0.00082608 / 0.00081887 CPU ] logAppend/three [ OK ] [ 0.00002901 / 0.00002849 CPU ] logAppend/two [ OK ] [ 0.00003217 / 0.00003169 CPU ] logAppend/one [ OK ] [ 0.00002468 / 0.00002399 CPU ] logGet/twoWithOffset [ OK ] [ 0.00003146 / 0.00003093 CPU ] logGet/two [ OK ] [ 0.00002976 / 0.00002917 CPU ] logGet/one [ OK ] [ 0.00002559 / 0.00002507 CPU ] logGet/emptyWithOffset [ OK ] [ 0.00007155 / 0.00007068 CPU ] logGet/empty_log [ OK ] [ 0.00000302 / 0.00000248 CPU ] logTermOf/snapshotTrailing [ OK ] [ 0.00003326 / 0.00003270 CPU ] logTermOf/withSnapshot [ OK ] [ 0.00002978 / 0.00002923 CPU ] logTermOf/two [ OK ] [ 0.00002971 / 0.00002914 CPU ] logTermOf/one [ OK ] [ 0.00002548 / 0.00002498 CPU ] logTermOf/snapshotLastIndex [ OK ] [ 0.00003333 / 0.00003335 CPU ] logTermOf/beyondLast [ OK ] [ 0.00000249 / 0.00000208 CPU ] logLastTerm/snapshot [ OK ] [ 0.00002435 / 0.00002417 CPU ] logLastTerm/empty [ OK ] [ 0.00000242 / 0.00000212 CPU ] logLastIndex/twoWithOffset [ OK ] [ 0.00002785 / 0.00002776 CPU ] logLastIndex/two [ OK ] [ 0.00002695 / 0.00002676 CPU ] logLastIndex/one [ OK ] [ 0.00002227 / 0.00002200 CPU ] logLastIndex/emptyWithOffset [ OK ] [ 0.00002347 / 0.00002339 CPU ] logLastIndex/empty [ OK ] [ 0.00000188 / 0.00000175 CPU ] logNumEntries/offsetNotEmpty [ OK ] [ 0.00002616 / 0.00002599 CPU ] logNumEntries/offset [ OK ] [ 0.00002817 / 0.00002780 CPU ] logNumEntries/wrapped [ OK ] [ 0.00002783 / 0.00002760 CPU ] logNumEntries/not_wrapped [ OK ] [ 0.00002853 / 0.00002791 CPU ] logNumEntries/empty [ OK ] [ 0.00000339 / 0.00000279 CPU ] flags/setMultipleClearSingle [ OK ] [ 0.00000777 / 0.00000667 CPU ] flags/setMultipleClearMultiple [ OK ] [ 0.00000282 / 0.00000227 CPU ] flags/setClear [ OK ] [ 0.00000211 / 0.00000189 CPU ] flags/empty [ OK ] [ 0.00000211 / 0.00000198 CPU ] ErrMsgWrapf/fullTruncate [ OK ] [ 0.00008602 / 0.00004914 CPU ] ErrMsgWrapf/partialTruncate [ OK ] [ 0.00004779 / 0.00004726 CPU ] ErrMsgWrapf/params [ OK ] [ 0.00005041 / 0.00004991 CPU ] ErrMsgWrapf/noParams [ OK ] [ 0.00004597 / 0.00004365 CPU ] ErrMsgPrintf/params [ OK ] [ 0.00011951 / 0.00011854 CPU ] ErrMsgPrintf/noParams [ OK ] [ 0.00000288 / 0.00000226 CPU ] configurationDecode/duplicatedID [ OK ] [ 0.00001021 / 0.00000969 CPU ] configurationDecode/badAddress [ OK ] [ 0.00000983 / 0.00000921 CPU ] configurationDecode/badVersion [ OK ] [ 0.00000661 / 0.00000608 CPU ] configurationDecode/oom heap-fault-delay=0, heap-fault-repeat=1 [ ERROR ] Error: test/unit/test_configuration.c:588: assertion failed: configurationDecode(&buf, &f->configuration) == RAFT_NOMEM (6 == 1) Error: child killed by signal 6 (Aborted) heap-fault-delay=1, heap-fault-repeat=1 [ ERROR ] Error: test/unit/test_configuration.c:588: assertion failed: configurationDecode(&buf, &f->configuration) == RAFT_NOMEM (6 == 1) Error: child killed by signal 6 (Aborted) heap-fault-delay=2, heap-fault-repeat=1 [ ERROR ] Error: test/unit/test_configuration.c:588: assertion failed: configurationDecode(&buf, &f->configuration) == RAFT_NOMEM (6 == 1) Error: child killed by signal 6 (Aborted) heap-fault-delay=3, heap-fault-repeat=1 [ ERROR ] Error: test/unit/test_configuration.c:588: assertion failed: configurationDecode(&buf, &f->configuration) == RAFT_NOMEM (6 == 1) Error: child killed by signal 6 (Aborted) configurationDecode/two_servers [ ERROR ] Error: test/unit/test_configuration.c:556: assertion failed: configurationDecode(&buf, &f->configuration) == 0 (6 == 0) Error: child killed by signal 6 (Aborted) configurationDecode/one_server [ ERROR ] Error: test/unit/test_configuration.c:532: assertion failed: configurationDecode(&buf, &f->configuration) == 0 (6 == 0) Error: child killed by signal 6 (Aborted) configurationEncode/oom [ OK ] [ 0.00001752 / 0.00001672 CPU ] configurationEncode/two_servers [ OK ] [ 0.00003614 / 0.00003538 CPU ] configurationEncode/one_server [ OK ] [ 0.00003855 / 0.00003788 CPU ] configurationRemove/oom [ OK ] [ 0.00003244 / 0.00003207 CPU ] configurationRemove/unknownId [ OK ] [ 0.00000235 / 0.00000198 CPU ] configurationRemove/middle [ OK ] [ 0.00008104 / 0.00008005 CPU ] configurationRemove/first [ OK ] [ 0.00004601 / 0.00004540 CPU ] configurationRemove/last [ OK ] [ 0.00002981 / 0.00002954 CPU ] configurationAdd/oom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00000559 / 0.00000541 CPU ] heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00000970 / 0.00000951 CPU ] configurationAdd/invalidRole [ OK ] [ 0.00000201 / 0.00000183 CPU ] configurationAdd/duplicateAddress [ OK ] [ 0.00001704 / 0.00001688 CPU ] configurationAdd/duplicateId [ OK ] [ 0.00002918 / 0.00002895 CPU ] configurationAdd/two [ OK ] [ 0.00003300 / 0.00003287 CPU ] configurationAdd/one [ OK ] [ 0.00001578 / 0.00001559 CPU ] configurationCopy/oom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00002154 / 0.00002140 CPU ] heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00002125 / 0.00002090 CPU ] heap-fault-delay=2, heap-fault-repeat=1 [ OK ] [ 0.00005240 / 0.00004587 CPU ] configurationCopy/two [ OK ] [ 0.00003626 / 0.00003610 CPU ] configurationGet/no_match [ OK ] [ 0.00001832 / 0.00001818 CPU ] configurationGet/match [ OK ] [ 0.00008918 / 0.00008781 CPU ] configurationIndexOfVoter/non_voting [ OK ] [ 0.00001718 / 0.00001697 CPU ] configurationIndexOfVoter/no_match [ OK ] [ 0.00002050 / 0.00001982 CPU ] configurationIndexOfVoter/match [ OK ] [ 0.00002957 / 0.00002919 CPU ] configurationIndexOf/no_match [ OK ] [ 0.00004845 / 0.00004696 CPU ] configurationIndexOf/match [ OK ] [ 0.00003590 / 0.00003589 CPU ] configurationVoterCount/filter [ OK ] [ 0.00003041 / 0.00002981 CPU ] configurationVoterCount/all_voters [ OK ] [ 0.00003829 / 0.00003354 CPU ] Compress/notCompressed [ OK ] [ 0.00000273 / 0.00000236 CPU ] Compress/isCompressed [ OK ] [ 0.00000582 / 0.00000553 CPU ] Compress/isCompressedNull [ OK ] [ 0.00000229 / 0.00000187 CPU ] Compress/isCompressedTooSmall [ OK ] [ 0.00000736 / 0.00000611 CPU ] Compress/decompress [ OK ] [ 0.00013347 / 0.00012277 CPU ] byteSha1/abcbd [ OK ] [ 0.00010075 / 0.00010038 CPU ] byteSha1/abcWithZeroLen [ OK ] [ 0.00010126 / 0.00010043 CPU ] byteSha1/abc [ OK ] [ 0.00008600 / 0.00008550 CPU ] byteGet64/success [ OK ] [ 0.00002464 / 0.00002414 CPU ] byteGetString/malformed [ OK ] [ 0.00000297 / 0.00000247 CPU ] byteGetString/success [ OK ] [ 0.00000266 / 0.00000234 CPU ] byteFlip/64 [ OK ] [ 0.00000239 / 0.00000217 CPU ] byteFlip/32 [ OK ] [ 0.00000232 / 0.00000208 CPU ] byteCrc32/invalid [ OK ] [ 0.00001343 / 0.00001290 CPU ] byteCrc32/valid [ OK ] [ 0.00000643 / 0.00000606 CPU ] 128 of 134 (96%) tests successful, 0 (0%) test skipped. FAIL test/unit/core (exit status: 1) FAIL: test/integration/core =========================== Running test suite with seed 0xdc76ca01... raft_transfer/preVote cluster-pre-vote=0, cluster-heartbeat=1 [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-pre-vote=0, cluster-heartbeat=100 [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-pre-vote=1, cluster-heartbeat=1 [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-pre-vote=1, cluster-heartbeat=100 [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_transfer/afterDemotion [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_transfer/autoSelectUpToDate [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_transfer/autoSelect [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_transfer/twice [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_transfer/unknownServer [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_transfer/expire [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_transfer/catchUp [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_transfer/upToDate [ ERROR ] Error: test/integration/test_transfer.c:77: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tick/request_vote_only_to_voters n_voting=2 [ ERROR ] Error: test/integration/test_tick.c:26: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tick/during_election [ ERROR ] Error: test/integration/test_tick.c:26: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tick/new_election [ ERROR ] Error: test/integration/test_tick.c:26: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tick/no_contact [ ERROR ] Error: test/integration/test_tick.c:26: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tick/not_voter n_voting=1 [ ERROR ] Error: test/integration/test_tick.c:26: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tick/electionTimerNotExpired [ ERROR ] Error: test/integration/test_tick.c:26: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tick/candidate [ ERROR ] Error: test/integration/test_tick.c:26: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tick/electionTimer [ ERROR ] Error: test/integration/test_tick.c:26: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_strerror/default [ OK ] [ 0.00004096 / 0.00003959 CPU ] raft_strerror/RAFT_IOERR [ OK ] [ 0.00002870 / 0.00001853 CPU ] raft_strerror/RAFT_BUSY [ OK ] [ 0.00001911 / 0.00001797 CPU ] raft_strerror/RAFT_NOCONNECTION [ OK ] [ 0.00002711 / 0.00002536 CPU ] raft_strerror/RAFT_TOOBIG [ OK ] [ 0.00003787 / 0.00003672 CPU ] raft_strerror/RAFT_NAMETOOLONG [ OK ] [ 0.00003855 / 0.00003918 CPU ] raft_strerror/RAFT_CANCELED [ OK ] [ 0.00003876 / 0.00003747 CPU ] raft_strerror/RAFT_CORRUPT [ OK ] [ 0.00001811 / 0.00001791 CPU ] raft_strerror/RAFT_CANTCHANGE [ OK ] [ 0.00001787 / 0.00001750 CPU ] raft_strerror/RAFT_CANTBOOTSTRAP [ OK ] [ 0.00001834 / 0.00001791 CPU ] raft_strerror/RAFT_SHUTDOWN [ OK ] [ 0.00001768 / 0.00001721 CPU ] raft_strerror/RAFT_LEADERSHIPLOST [ OK ] [ 0.00001802 / 0.00001769 CPU ] raft_strerror/RAFT_NOTLEADER [ OK ] [ 0.00004030 / 0.00003884 CPU ] raft_strerror/RAFT_MALFORMED [ OK ] [ 0.00002005 / 0.00001960 CPU ] raft_strerror/RAFT_BADROLE [ OK ] [ 0.00001839 / 0.00001837 CPU ] raft_strerror/RAFT_DUPLICATEADDRESS [ OK ] [ 0.00001869 / 0.00001840 CPU ] raft_strerror/RAFT_DUPLICATEID [ OK ] [ 0.00001933 / 0.00001888 CPU ] raft_strerror/RAFT_BADID [ OK ] [ 0.00001881 / 0.00001851 CPU ] raft_strerror/RAFT_NOMEM [ OK ] [ 0.00001854 / 0.00001834 CPU ] raft_start/singleVotingNotUs [ ERROR ] Error: test/integration/test_start.c:199: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_start/singleVotingSelfElect [ ERROR ] Error: test/integration/test_start.c:186: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_start/twoEntries [ ERROR ] Error: test/integration/test_start.c:166: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_start/noEntries [ ERROR ] Error: test/integration/test_start.c:133: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_start/oneSnapshotAndSomeFollowUpEntries [ ERROR ] Error: test/integration/test_start.c:110: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_start/oneSnapshotAndNoEntries [ ERROR ] Error: test/integration/test_start.c:76: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/newTermWhileInstalling [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/unavailableDiscardsSnapshot [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/snapshotBlocksCandidate [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/takeSnapshotFail fsm-version=1 [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) fsm-version=2 [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/takeSnapshotSnapshotPutFail fsm-version=1 [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) fsm-version=2 [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/takeSnapshotAppendEntries fsm-version=1 [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) fsm-version=2 [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installSnapshotDuringEntriesWrite [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/receiveAppendEntriesWhileInstalling [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installSnapshotHeartBeats [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installMultipleTimeOutAppendAfter [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installMultipleTimeOut [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installOneTimeOutAppendAfter [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/noSnapshotInstallToKilled [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installOneDisconnectedDuringOperationReconnects [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installOneDisconnectedFromBeginningReconnects [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installOneTimeOut [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) snapshot/installOne [ ERROR ] Error: test/integration/test_snapshot.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/noLeaderAfterAppending [ ERROR ] Error: test/integration/test_replication.c:1225: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/failPersistBarrier [ ERROR ] Error: test/integration/test_replication.c:1205: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/lastStoredLaggingBehindCommitIndex [ ERROR ] Error: test/integration/test_replication.c:1150: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/newTermWhileAppending [ ERROR ] Error: test/integration/test_replication.c:1113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/diskWriteFailure [ ERROR ] Error: test/integration/test_replication.c:1091: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/resultRetry [ ERROR ] Error: test/integration/test_replication.c:1065: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/resultHigherTerm [ ERROR ] Error: test/integration/test_replication.c:1027: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/resultLowerTerm [ ERROR ] Error: test/integration/test_replication.c:994: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/resultNotLeader [ ERROR ] Error: test/integration/test_replication.c:967: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvCandidateHigherTerm [ ERROR ] Error: test/integration/test_replication.c:931: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvCandidateSameTerm [ ERROR ] Error: test/integration/test_replication.c:885: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvMatch_last_snapshot [ ERROR ] Error: test/integration/test_replication.c:858: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvSkip [ ERROR ] Error: test/integration/test_replication.c:808: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvPrevIndexConflict [ ERROR ] Error: test/integration/test_replication.c:791: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvRollbackConfigurationToSnapshot [ ERROR ] Error: test/integration/test_replication.c:751: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvRollbackConfigurationToPrevious [ ERROR ] Error: test/integration/test_replication.c:681: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvRollbackConfigurationToInitial [ ERROR ] Error: test/integration/test_replication.c:615: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvPrevLogTermMismatch [ ERROR ] Error: test/integration/test_replication.c:573: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvMissingEntries [ ERROR ] Error: test/integration/test_replication.c:542: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvStaleTerm [ ERROR ] Error: test/integration/test_replication.c:504: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/recvTwice [ ERROR ] Error: test/integration/test_replication.c:479: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/sendIoError [ SKIP ] replication/sendOom heap-fault-delay=5, heap-fault-repeat=1 [ SKIP ] replication/sendDisconnectPipeline [ ERROR ] Error: test/integration/test_replication.c:402: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/sendDisconnect [ ERROR ] Error: test/integration/test_replication.c:370: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/sendPipeline [ ERROR ] Error: test/integration/test_replication.c:326: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/sendProbe [ ERROR ] Error: test/integration/test_replication.c:269: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/skipIdle [ ERROR ] Error: test/integration/test_replication.c:250: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/sendSkipHeartbeat [ ERROR ] Error: test/integration/test_replication.c:215: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/sendFollowupHeartbeat [ ERROR ] Error: test/integration/test_replication.c:171: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/receiveFlags [ ERROR ] Error: test/integration/test_replication.c:137: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/sendInitialHeartbeat [ ERROR ] Error: test/integration/test_replication.c:101: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_recover/busy [ ERROR ] Error: test/integration/test_recover.c:47: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_remove/badId [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_remove/inProgress [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_remove/notLeader [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_remove/selfThreeNodeClusterReplicate [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_remove/self [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_remove/committed [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_add/duplicateId [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_add/busy [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_add/notLeader [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_add/committed [ ERROR ] Error: test/integration/test_membership.c:24: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_init/oom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00002387 / 0.00002205 CPU ] heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00002511 / 0.00002388 CPU ] raft_init/nullFsmAndIo [ OK ] [ 0.00007923 / 0.00007877 CPU ] raft_init/fsmVersionNotSet [ OK ] [ 0.00005034 / 0.00002566 CPU ] raft_init/ioVersionNotSet [ OK ] [ 0.00002061 / 0.00002001 CPU ] raft_heap/aligned_alloc [ OK ] [ 0.00005089 / 0.00003875 CPU ] raft_heap/realloc [ OK ] [ 0.00006406 / 0.00006212 CPU ] raft_heap/calloc [ OK ] [ 0.00004394 / 0.00004358 CPU ] raft_heap/malloc [ OK ] [ 0.00006374 / 0.00006281 CPU ] raft_fixture_step_until_applied/two [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_step_until_applied/one [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_elect/again [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_elect/change [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_elect/second [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_elect/first [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_step/deliver [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_step/flushSend [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_step/electionTimeout [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_fixture_step/tick [ ERROR ] Error: test/integration/test_fixture.c:49: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/startElectionWithUnpersistedEntries cluster-n=4, cluster-n-voting=3 [ ERROR ] Error: test/integration/test_election.c:815: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/preVoteNoStaleVotes cluster-n=3 [ ERROR ] Error: test/integration/test_election.c:722: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/preVoteWithcandidateCrash cluster-n=3 [ ERROR ] Error: test/integration/test_election.c:627: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/preVote [ ERROR ] Error: test/integration/test_election.c:585: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/ioErrorPersistVote [ SKIP ] election/ioErrorSendVoteRequest [ SKIP ] election/ioErrorConvert delay=0 [ SKIP ] delay=1 [ SKIP ] election/receiveRejectResult cluster-n=5 [ ERROR ] Error: test/integration/test_election.c:486: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/rejectIfNotVoter cluster-n=3, cluster-n-voting=2 [ ERROR ] Error: test/integration/test_election.c:450: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/rejectIfLastIndexIsLower [ ERROR ] Error: test/integration/test_election.c:407: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/rejectIfLastTermIsLower [ ERROR ] Error: test/integration/test_election.c:368: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/rejectIfAlreadyVoted cluster-n=3 [ ERROR ] Error: test/integration/test_election.c:327: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/rejectIfHasLeader cluster-n=3 [ ERROR ] Error: test/integration/test_election.c:299: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/rejectIfHigherTerm [ ERROR ] Error: test/integration/test_election.c:273: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/waitQuorum cluster-n=5 [ ERROR ] Error: test/integration/test_election.c:239: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/grantIfLastIndexIsHigher [ ERROR ] Error: test/integration/test_election.c:217: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/grantIfLastIndexIsSame [ ERROR ] Error: test/integration/test_election.c:190: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/grantAgain [ ERROR ] Error: test/integration/test_election.c:145: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/twoVoters [ ERROR ] Error: test/integration/test_election.c:115: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_digest/bootstrapServerId [ OK ] [ 0.00008168 / 0.00007997 CPU ] raft_bootstrap/busy [ ERROR ] Error: test/integration/test_bootstrap.c:48: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_barrier/cb [ ERROR ] Error: test/integration/test_barrier.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/demoteLeaderToSpare [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/demoteLeader [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/demoteToStandBy [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/promoteUnresponsive [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/leadershipLost [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/changeRequestAlreadyInProgress [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/alreadyHasRole [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/badRole [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/unknownId [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/notLeader [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/promoteToStandBy [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/changeIsImmediate [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/promoteNewRound [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/promoteCatchUp [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_assign/promoteUpToDate [ ERROR ] Error: test/integration/test_assign.c:113: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_apply/leadershipLost [ ERROR ] Error: test/integration/test_apply.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_apply/notLeader [ ERROR ] Error: test/integration/test_apply.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_apply/two [ ERROR ] Error: test/integration/test_apply.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) raft_apply/first [ ERROR ] Error: test/integration/test_apply.c:20: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) 29 of 163 (18%) tests successful, 6 (4%) test skipped. FAIL test/integration/core (exit status: 1) FAIL: test/fuzzy/core ===================== Running test suite with seed 0xeb42e539... replication/partitioned cluster-n=3 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/noQuorum cluster-n=3 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/availability cluster-n=3 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) replication/appendEntries cluster-n=3 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7 [ ERROR ] Error: test/fuzzy/test_replication.c:28: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) membership/removeVoting cluster-n=3 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) membership/addVoting cluster-n=3 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) membership/addNonVoting cluster-n=3 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5 [ ERROR ] Error: test/fuzzy/test_membership.c:29: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) liveness/networkDisconnect cluster-n=3, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_liveness.c:85: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=3, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_liveness.c:85: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_liveness.c:85: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_liveness.c:85: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/noQuorum cluster-n=3, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=3, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/changeReviveOutdated cluster-n=3, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=3, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/change cluster-n=3, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=3, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) election/win cluster-n=3, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=3, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=4, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=5, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7, cluster-pre-vote=0 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) cluster-n=7, cluster-pre-vote=1 [ ERROR ] Error: test/fuzzy/test_election.c:30: assertion failed: rc == 0 (6 == 0) Error: child killed by signal 6 (Aborted) 0 of 57 (0%) tests successful, 0 (0%) test skipped. FAIL test/fuzzy/core (exit status: 1) FAIL: test/integration/uv ========================= Running test suite with seed 0xb2ec807d... snapshot_truncate/snapshotThenTruncate [ ERROR ] src/uv_append.c:598 - enqueue 4 entries src/uv_prepare.c:166 - create open segment open-1 src/uv_append.c:630 - set uv->append_next_index 5 src/uv_append.c:598 - enqueue 4 entries src/uv_append.c:630 - set uv->append_next_index 9 src/uv_append.c:598 - enqueue 4 entries src/uv_append.c:630 - set uv->append_next_index 13 src/uv_snapshot.c:622 - put snapshot at 6, keeping 8192 src/uv_append.c:815 - UvBarrier uv->append_next_index:13 src/uv_truncate.c:173 - uv truncate 8 src/uv_append.c:815 - UvBarrier uv->append_next_index:8 src/uv_prepare.c:232 - completed creation of open-1 src/uv_prepare.c:166 - create open segment open-2 src/uv_finalize.c: 39 - finalize open-1 into 0000000000000001-0000000000000004 src/uv_prepare.c:232 - completed creation of open-2 src/uv_prepare.c:166 - create open segment open-3 src/uv_finalize.c: 78 - uv finalize after work segment 0x55b9923350a0 cb status:0 src/uv_finalize.c: 93 - unblock barrier or close src/uv.c:189 - uv maybe fire close cb src/uv_finalize.c: 39 - finalize open-2 into 0000000000000005-0000000000000008 src/uv_prepare.c:232 - completed creation of open-3 src/uv_prepare.c:166 - create open segment open-4 src/uv_finalize.c: 78 - uv finalize after work segment 0x55b9923339c0 cb status:0 src/uv_finalize.c: 93 - unblock barrier or close src/uv.c:189 - uv maybe fire close cb src/uv_finalize.c: 39 - finalize open-3 into 0000000000000009-0000000000000012 src/uv_prepare.c:232 - completed creation of open-4 src/uv_prepare.c:166 - create open segment open-5 src/uv_finalize.c: 78 - uv finalize after work segment 0x55b992334520 cb status:0 src/uv_finalize.c: 93 - unblock barrier or close src/uv.c:189 - uv maybe fire close cb src/uv_prepare.c:232 - completed creation of open-5 src/uv_snapshot.c:505 - snapshot write start src/uv_snapshot.c:508 - snapshot write end 0 src/uv_list.c: 92 - segment 0000000000000001-0000000000000004 src/uv_list.c: 92 - segment 0000000000000005-0000000000000008 src/uv_list.c: 92 - segment 0000000000000009-0000000000000012 src/uv_list.c: 92 - segment open-4 src/uv_list.c: 92 - segment open-5 src/uv_list.c: 97 - ignore snapshot-1-6-1005566117 src/uv_list.c: 82 - snapshot snapshot-1-6-1005566117.meta src/uv_truncate.c:125 - uv truncate barrier cb src/uv_truncate.c:144 - set truncate work src/uv_append.c:896 - UvUnblock triggered barrier request callback. src/uv_truncate.c: 26 - uv truncate work cb src/uv_list.c: 92 - segment 0000000000000001-0000000000000004 src/uv_list.c: 92 - segment 0000000000000005-0000000000000008 src/uv_list.c: 92 - segment 0000000000000009-0000000000000012 src/uv_list.c: 92 - segment open-4 src/uv_list.c: 92 - segment open-5 src/uv_list.c: 97 - ignore snapshot-1-6-1005566117 src/uv_list.c: 82 - snapshot snapshot-1-6-1005566117.meta src/uv_segment.c:1087 - truncate 5-8 at 8 src/uv_truncate.c: 93 - uv truncate work cb ok src/uv_truncate.c:110 - uv truncate after work cb status:0 src/uv_truncate.c:115 - clear truncate work src/uv_append.c:901 - UvUnblock queue empty src/uv_append.c:930 - uv barrier close src/uv.c:189 - uv maybe fire close cb src/uv.c:189 - uv maybe fire close cb src/uv.c:189 - uv maybe fire close cb src/uv.c:189 - uv maybe fire close cb src/uv.c:189 - uv maybe fire close cb src/uv_list.c: 92 - segment 0000000000000001-0000000000000004 src/uv_list.c: 92 - segment 0000000000000005-0000000000000007 src/uv_list.c: 97 - ignore snapshot-1-6-1005566117 src/uv_list.c: 82 - snapshot snapshot-1-6-1005566117.meta Error: test/integration/test_uv_truncate_snapshot.c:244: assertion failed: _ret == 0 (6 == 0) Error: child killed by signal 6 (Aborted) truncate/closingMultiple [ OK ] [ 0.00407910 / 0.00431193 CPU ] truncate/closing [ OK ] [ 0.00417731 / 0.00430895 CPU ] truncate/multiplePending [ OK ] [ 0.11447962 / 0.02222023 CPU ] truncate/pendingAppend [ OK ] [ 0.10021510 / 0.01457159 CPU ] truncate/partialSegment [ OK ] [ 0.09317212 / 0.01447371 CPU ] truncate/sameAsLastIndex [ OK ] [ 0.08518391 / 0.01512824 CPU ] truncate/wholeSegment [ OK ] [ 0.06536535 / 0.01403008 CPU ] snapshot_put/installMultipleAppendEntriesInBetween [ OK ] [ 0.03763012 / 0.01381621 CPU ] snapshot_put/installMultipleWithoutPreviousEntries [ OK ] [ 0.00109858 / 0.00139070 CPU ] snapshot_put/installWithoutPreviousEntries [ OK ] [ 0.00095619 / 0.00116358 CPU ] snapshot_put/install [ OK ] [ 0.02225517 / 0.00797336 CPU ] snapshot_put/entriesMoreThanTrailing [ OK ] [ 0.07941501 / 0.00371128 CPU ] snapshot_put/entriesLessThanTrailing [ OK ] [ 0.06657139 / 0.00356807 CPU ] snapshot_put/first [ ERROR ] src/uv_snapshot.c:622 - put snapshot at 1, keeping 10 src/uv_append.c:815 - UvBarrier uv->append_next_index:1 src/uv_snapshot.c:505 - snapshot write start src/uv_snapshot.c:508 - snapshot write end 0 src/uv_list.c: 97 - ignore snapshot-1-1-1005567564 src/uv_list.c: 82 - snapshot snapshot-1-1-1005567564.meta src/uv_append.c:901 - UvUnblock queue empty src/uv_list.c: 97 - ignore snapshot-1-1-1005567564 src/uv_list.c: 82 - snapshot snapshot-1-1-1005567564.meta Error: test/integration/test_uv_snapshot_put.c:49: assertion failed: status == 0 (6 == 0) Error: child killed by signal 6 (Aborted) tcp_listen/handshake n=8 [ OK ] [ 0.00028881 / 0.00025406 CPU ] n=16 [ OK ] [ 0.00024569 / 0.00021940 CPU ] n=24 [ OK ] [ 0.00039091 / 0.00032058 CPU ] n=32 [ OK ] [ 0.00021243 / 0.00019204 CPU ] tcp_listen/closeBeforeHandshake [ OK ] [ 0.00020318 / 0.00018513 CPU ] tcp_listen/pending [ OK ] [ 0.00014265 / 0.00012626 CPU ] tcp_listen/oom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00020803 / 0.00018601 CPU ] heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00021668 / 0.00019444 CPU ] heap-fault-delay=3, heap-fault-repeat=1 [ OK ] [ 0.00025036 / 0.00021673 CPU ] tcp_listen/peerAbort n=8 [ OK ] [ 0.00028392 / 0.00021109 CPU ] n=16 [ OK ] [ 0.00022777 / 0.00019896 CPU ] n=24 [ OK ] [ 0.00021909 / 0.00019333 CPU ] n=32 [ OK ] [ 0.00019798 / 0.00017138 CPU ] tcp_listen/badProtocol [ OK ] [ 0.00022363 / 0.00017391 CPU ] tcp_listen/resolveFailure [ OK ] [ 0.00001769 / 0.00001692 CPU ] tcp_listen/cannotBindSecond [ OK ] [ 0.00008849 / 0.00008802 CPU ] tcp_listen/cannotBindFirst [ OK ] [ 0.00006221 / 0.00006179 CPU ] tcp_listen/alreadyBound [ OK ] [ 0.00016336 / 0.00016255 CPU ] tcp_listen/secondOfTwo [ OK ] [ 0.00022018 / 0.00018657 CPU ] tcp_listen/firstOfTwo [ OK ] [ 0.00021224 / 0.00018353 CPU ] tcp_listen/invalidAddress address=500.1.2.3:9000, bind-address= [ OK ] [ 0.00017658 / 0.00017631 CPU ] address=500.1.2.3:9000, bind-address=500.1.2.3:9000 [ OK ] [ 0.00015872 / 0.00015841 CPU ] address=500.1.2.3:9000, bind-address=not-existing:9000 [ OK ] [ 3.87283102 / 0.00108097 CPU ] address=500.1.2.3:9000, bind-address=192.0.2.0:9000 [ OK ] [ 0.00013009 / 0.00012976 CPU ] address=not-existing:9000, bind-address= [ OK ] [ 3.99736761 / 0.00115402 CPU ] address=not-existing:9000, bind-address=500.1.2.3:9000 [ OK ] [ 0.00018521 / 0.00018482 CPU ] address=not-existing:9000, bind-address=not-existing:9000 [ OK ] [ 3.99604897 / 0.00139265 CPU ] address=not-existing:9000, bind-address=192.0.2.0:9000 [ OK ] [ 0.00029009 / 0.00028918 CPU ] address=192.0.2.0:9000, bind-address= [ OK ] [ 0.00010569 / 0.00010529 CPU ] address=192.0.2.0:9000, bind-address=500.1.2.3:9000 [ OK ] [ 0.00037617 / 0.00035830 CPU ] address=192.0.2.0:9000, bind-address=not-existing:9000 [ OK ] [ 3.99226893 / 0.00126701 CPU ] address=192.0.2.0:9000, bind-address=192.0.2.0:9000 [ OK ] [ 0.00013301 / 0.00013238 CPU ] tcp_listen/success address=127.0.0.1:9000, bind-address= [ OK ] [ 0.00027785 / 0.00024106 CPU ] address=127.0.0.1:9000, bind-address=127.0.0.1:9000 [ OK ] [ 0.00024643 / 0.00021017 CPU ] address=127.0.0.1:9000, bind-address=localhost:9000 [ OK ] [ 0.00049575 / 0.00045881 CPU ] address=127.0.0.1:9000, bind-address=:9000 [ OK ] [ 0.00024628 / 0.00021854 CPU ] address=127.0.0.1:9000, bind-address=0.0.0.0:9000 [ OK ] [ 0.00024043 / 0.00021290 CPU ] address=localhost:9000, bind-address= [ OK ] [ 0.00080724 / 0.00076337 CPU ] address=localhost:9000, bind-address=127.0.0.1:9000 [ OK ] [ 0.00026778 / 0.00023806 CPU ] address=localhost:9000, bind-address=localhost:9000 [ OK ] [ 0.00045957 / 0.00042514 CPU ] address=localhost:9000, bind-address=:9000 [ OK ] [ 0.00023066 / 0.00020421 CPU ] address=localhost:9000, bind-address=0.0.0.0:9000 [ OK ] [ 0.00026677 / 0.00023852 CPU ] tcp_connect/closeDuringSecondConnect [ OK ] [ 0.00056882 / 0.00087961 CPU ] tcp_connect/closeDuringConnectAbort [ OK ] [ 0.00052942 / 0.00067390 CPU ] tcp_connect/closeDuringDnsLookupAbort [ OK ] [ 0.00057376 / 0.00080340 CPU ] tcp_connect/closeDuringHandshake [ SKIP ] tcp_connect/closeDuringDnsLookup [ OK ] [ 0.00060401 / 0.00086051 CPU ] tcp_connect/closeImmediately [ OK ] [ 0.00041436 / 0.00064706 CPU ] tcp_connect/oom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00001145 / 0.00001080 CPU ] heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00001615 / 0.00001542 CPU ] heap-fault-delay=2, heap-fault-repeat=1 [ OK ] [ 0.00001789 / 0.00001713 CPU ] tcp_connect/refused [ OK ] [ 0.00050974 / 0.00071123 CPU ] tcp_connect/secondIP [ OK ] [ 0.00050371 / 0.00058896 CPU ] tcp_connect/firstIP [ OK ] [ 0.00042602 / 0.00051499 CPU ] tcp_connect/connectByName [ OK ] [ 0.00081133 / 0.00089290 CPU ] tcp_connect/first [ OK ] [ 0.00051689 / 0.00059953 CPU ] set_term/metadataTwoIsGreater [ OK ] [ 0.02346517 / 0.00041714 CPU ] set_term/metadataOneIsGreater [ OK ] [ 0.02064428 / 0.00040205 CPU ] set_term/metadataOneExists [ OK ] [ 0.01747042 / 0.00041948 CPU ] set_term/fourth [ OK ] [ 0.00022915 / 0.00022817 CPU ] set_term/third [ OK ] [ 0.00020745 / 0.00020659 CPU ] set_term/second [ OK ] [ 0.00020106 / 0.00019916 CPU ] set_term/first [ OK ] [ 0.00012283 / 0.00012194 CPU ] send/closeDuringConnection [ OK ] [ 0.00065742 / 0.00098336 CPU ] send/closeDuringWrite [ OK ] [ 0.00074206 / 0.00109816 CPU ] send/oomAsync heap-fault-delay=2, heap-fault-repeat=1 [ OK ] [ 0.00065867 / 0.00092968 CPU ] send/oom heap-fault-delay=0, heap-fault-repeat=1 [ OK ] [ 0.00002330 / 0.00002208 CPU ] heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00003565 / 0.00003463 CPU ] heap-fault-delay=2, heap-fault-repeat=1 [ OK ] [ 0.00003705 / 0.00003596 CPU ] heap-fault-delay=3, heap-fault-repeat=1 [ OK ] [ 0.00003730 / 0.00003618 CPU ] heap-fault-delay=4, heap-fault-repeat=1 [ OK ] [ 0.00003946 / 0.00003845 CPU ] send/reconnectAfterMultipleWriteErrors [ OK ] [ 0.00121510 / 0.00148829 CPU ] send/reconnectAfterWriteError [ OK ] [ 0.00103215 / 0.00130627 CPU ] send/evictOldPending [ OK ] [ 0.00088297 / 0.00128799 CPU ] send/badMessage [ OK ] [ 0.00003634 / 0.00003520 CPU ] send/changeToUnconnectedAddress [ OK ] [ 0.00105302 / 0.00140724 CPU ] send/badAddress [ OK ] [ 0.00072067 / 0.00120743 CPU ] send/noConnection [ OK ] [ 0.00051786 / 0.00086827 CPU ] send/installSnapshot [ OK ] [ 0.00067963 / 0.00100528 CPU ] send/appendEntriesResult [ OK ] [ 0.00058109 / 0.00080465 CPU ] send/heartbeat [ OK ] [ 0.00082545 / 0.00107648 CPU ] send/appendEntries [ OK ] [ 0.00075581 / 0.00102032 CPU ] send/voteResult [ OK ] [ 0.00071345 / 0.00092582 CPU ] send/parallel [ OK ] [ 0.00067725 / 0.00104832 CPU ] send/second [ OK ] [ 0.00067051 / 0.00096069 CPU ] send/first [ OK ] [ 0.00075797 / 0.00108879 CPU ] recv/closeAfterAppendEntriesHeader [ SKIP ] recv/closeBeforeAccept [ OK ] [ 0.00018817 / 0.00015598 CPU ] recv/badType [ OK ] [ 0.00020810 / 0.00014443 CPU ] recv/badSize [ OK ] [ 0.00022572 / 0.00015391 CPU ] recv/badProtocol [ OK ] [ 0.00020071 / 0.00013710 CPU ] recv/timeoutNow [ OK ] [ 0.00056414 / 0.00093034 CPU ] recv/installSnapshot [ ERROR ] src/uv_recv.c:267 - decode message: encoded data is malformed src/uv.c:189 - uv maybe fire close cb Error: test/integration/test_uv_recv.c:397: uv_run: condition not met in 20 iterations Error: child killed by signal 6 (Aborted) recv/appendEntriesResult [ OK ] [ 0.00069927 / 0.00111471 CPU ] recv/heartbeat [ OK ] [ 0.00074734 / 0.00113562 CPU ] recv/appendEntries [ OK ] [ 0.00061848 / 0.00098846 CPU ] recv/requestVoteResult [ OK ] [ 0.00062546 / 0.00089644 CPU ] recv/second [ OK ] [ 0.00080699 / 0.00117974 CPU ] recv/first [ OK ] [ 0.00061717 / 0.00096768 CPU ] recover/newConfiguration [ OK ] [ 0.00042333 / 0.00042159 CPU ] load/openSegmentWithBadFormat [ OK ] [ 0.05900911 / 0.00248433 CPU ] load/openSegmentWithZeroFormatAndThenData [ OK ] [ 0.05382194 / 0.00204371 CPU ] load/openSegmentWithNoAccessPermission [ OK ] [ 0.06790470 / 0.00247343 CPU ] load/closedSegmentWithBadFormat [ OK ] [ 0.01998517 / 0.00040785 CPU ] load/emptyClosedSegment [ OK ] [ 0.01945647 / 0.00034363 CPU ] load/closedSegmentWithBadIndex [ OK ] [ 0.09307933 / 0.00349175 CPU ] load/closedSegmentWithCorruptedBatchData [ OK ] [ 0.05406988 / 0.00205651 CPU ] load/closedSegmentWithCorruptedBatchHeader [ OK ] [ 0.04883436 / 0.00245870 CPU ] load/openSegmentWithIncompleteBatchData [ OK ] [ 0.05471388 / 0.00226117 CPU ] load/openSegmentWithIncompleteBatchHeader [ OK ] [ 0.04802409 / 0.00188830 CPU ] load/openSegmentWithIncompletePreamble [ OK ] [ 0.04827115 / 0.00205171 CPU ] load/openSegmentWithIncompleteFormat [ OK ] [ 0.02006616 / 0.00043918 CPU ] load/closedSegmentWithEntriesPastSnapshot [ ERROR ] src/uv_list.c: 92 - segment 0000000000000006-0000000000000006 src/uv_list.c: 97 - ignore snapshot-1-4-1005765883 src/uv_list.c: 82 - snapshot snapshot-1-4-1005765883.meta Error: test/integration/test_uv_load.c:1593: assertion failed: _rv == RAFT_CORRUPT (6 == 12) Error: child killed by signal 6 (Aborted) load/nonContiguousClosedSegments [ ERROR ] src/uv_list.c: 92 - segment 0000000000000001-0000000000000001 src/uv_list.c: 92 - segment 0000000000000004-0000000000000006 src/uv_list.c: 97 - ignore snapshot-1-4-1005766231 src/uv_list.c: 82 - snapshot snapshot-1-4-1005766231.meta Error: test/integration/test_uv_load.c:1566: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/closedSegmentsWithSnapshotSecondLastSegmentCorrupt [ ERROR ] Error: test/integration/test_uv_load.c:1528: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/closedSegmentsWithSnapshotLastSegmentCorruptOpenSegment[ ERROR ] Error: test/integration/test_uv_load.c:1498: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/closedSegmentsWithSnapshotLastSegmentCorruptOpenSegmentWithAutoRecovery[ ERROR ] Error: test/integration/test_uv_load.c:1462: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/closedSegmentsWithSnapshotLastSegmentCorruptAutoRecovery[ ERROR ] Error: test/integration/test_uv_load.c:1426: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/closedSegmentsWithSnapshotLastSegmentCorrupt [ ERROR ] Error: test/integration/test_uv_load.c:1394: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/closedSegmentsOverlappingWithSnapshot [ ERROR ] src/uv_list.c: 92 - segment 0000000000000001-0000000000000001 src/uv_list.c: 92 - segment 0000000000000002-0000000000000003 src/uv_list.c: 92 - segment 0000000000000004-0000000000000006 src/uv_list.c: 97 - ignore snapshot-1-4-1005767860 src/uv_list.c: 82 - snapshot snapshot-1-4-1005767860.meta Error: test/integration/test_uv_load.c:1373: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/corruptLastOpenSegmentClosedSegmentsWithAutoRecovery [ OK ] [ 0.12352446 / 0.00405008 CPU ] load/corruptLastOpenSegmentClosedSegments [ OK ] [ 0.11491814 / 0.00432911 CPU ] load/corruptOpenSegmentsClosedSegmentsWithAutoRecovery [ OK ] [ 0.12524168 / 0.00508722 CPU ] load/corruptOpenSegmentsClosedSegments [ OK ] [ 0.12687088 / 0.00443750 CPU ] load/corruptOpenSegmentClosedSegmentsWithAutoRecovery [ OK ] [ 0.08100853 / 0.00349469 CPU ] load/corruptOpenSegmentClosedSegments [ OK ] [ 0.08192313 / 0.00372477 CPU ] load/corruptOpenSegmentClosedSegmentsSnapshotPresent [ ERROR ] Error: test/integration/test_uv_load.c:1193: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/corruptOpenSegmentClosedSegmentsSnapshotPresentWithAutoRecovery[ ERROR ] Error: test/integration/test_uv_load.c:1161: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/corruptOpenSegmentClosedSegmentSnapshotPresentWithAutoRecovery[ ERROR ] Error: test/integration/test_uv_load.c:1123: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/corruptOpenSegmentClosedSegmentSnapshotPresent [ ERROR ] Error: test/integration/test_uv_load.c:1095: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/corruptOpenSegmentNoClosedSegmentsSnapshotPresentWithAutoRecovery[ ERROR ] Error: test/integration/test_uv_load.c:1067: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/corruptOpenSegmentNoClosedSegmentsSnapshotPresent [ ERROR ] Error: test/integration/test_uv_load.c:1040: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/openSegmentNoClosedSegmentsSnapshotPresent [ ERROR ] Error: test/integration/test_uv_load.c:1017: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/openSegmentWithEntriesBehindSnapshotAutoRecovery [ ERROR ] src/uv_list.c: 92 - segment 0000000000000001-0000000000000001 src/uv_list.c: 92 - segment open-1 src/uv_list.c: 97 - ignore snapshot-1-3-1005770531 src/uv_list.c: 82 - snapshot snapshot-1-3-1005770531.meta Error: test/integration/test_uv_load.c:996: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/openSegmentWithEntriesBehindSnapshot [ ERROR ] src/uv_list.c: 92 - segment 0000000000000001-0000000000000001 src/uv_list.c: 92 - segment open-1 src/uv_list.c: 97 - ignore snapshot-1-3-1005770834 src/uv_list.c: 82 - snapshot snapshot-1-3-1005770834.meta Error: test/integration/test_uv_load.c:969: assertion failed: _rv == RAFT_CORRUPT (6 == 12) Error: child killed by signal 6 (Aborted) load/closedSegmentWithInconsistentFilenameAutoRecovery [ OK ] [ 0.05681519 / 0.00270632 CPU ] load/closedSegmentWithInconsistentFilename [ OK ] [ 0.04896906 / 0.00245639 CPU ] load/openSegmentWithEntriesPastSnapshot [ ERROR ] src/uv_list.c: 92 - segment 0000000000000001-0000000000000001 src/uv_list.c: 92 - segment open-1 src/uv_list.c: 97 - ignore snapshot-1-2-1005771260 src/uv_list.c: 82 - snapshot snapshot-1-2-1005771260.meta Error: test/integration/test_uv_load.c:909: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/closedSegmentWithEntriesBehindSnapshot [ ERROR ] src/uv_list.c: 92 - segment 0000000000000001-0000000000000001 src/uv_list.c: 97 - ignore snapshot-1-2-1005771556 src/uv_list.c: 82 - snapshot snapshot-1-2-1005771556.meta Error: test/integration/test_uv_load.c:883: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/orphanedSnapshotFiles [ ERROR ] src/uv_list.c: 97 - ignore snapshot-2-16-1005771829 src/uv_list.c: 82 - snapshot snapshot-2-16-1005771829.meta Error: test/integration/test_uv_load.c:856: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/emptySnapshot [ ERROR ] Error: test/integration/test_uv_load.c:801: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/manySnapshots [ ERROR ] Error: test/integration/test_uv_load.c:763: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/onlyOneSnapshot [ ERROR ] src/uv_list.c: 97 - ignore snapshot-1-1-1005772601 src/uv_list.c: 82 - snapshot snapshot-1-1-1005772601.meta Error: test/integration/test_uv_load.c:731: assertion failed: _rv == 0 (6 == 0) Error: child killed by signal 6 (Aborted) load/openSegment [ OK ] [ 0.05541627 / 0.00229550 CPU ] load/twoOpenSegmentsFirstCorrupt [ OK ] [ 0.07052755 / 0.00221551 CPU ] load/twoOpenSegmentsFirstCorruptAutoRecovery [ OK ] [ 0.05179622 / 0.00229808 CPU ] load/secondOpenSegmentIsAllZeros [ OK ] [ 0.04840933 / 0.00246250 CPU ] load/twoOpenSegments [ OK ] [ 0.09303569 / 0.00335942 CPU ] load/openSegmentWithIncompleteFirstBatch [ OK ] [ 0.05325279 / 0.00204655 CPU ] load/openSegmentWithIncompleteBatch [ OK ] [ 0.05643837 / 0.00267136 CPU ] load/openSegmentWithNonZeroData [ OK ] [ 0.05704415 / 0.00227050 CPU ] load/bothOpenAndClosedSegments [ OK ] [ 0.13627549 / 0.00518218 CPU ] load/openSegmentWithTrailingZeros [ OK ] [ 0.02385161 / 0.00041209 CPU ] load/emptyOpenSegment [ OK ] [ 0.01250767 / 0.00057811 CPU ] load/removeUnusableFiles filename=tmp-0000000001221212-0000000001221217 [ OK ] [ 0.02187261 / 0.00054429 CPU ] filename=tmp-snapshot-15-8260687-512469866 [ OK ] [ 0.02423017 / 0.00060596 CPU ] filename=snapshot-525-43326736-880259052 [ OK ] [ 0.02551867 / 0.00044965 CPU ] filename=snapshot-999-13371337-880259052.meta [ OK ] [ 0.01844353 / 0.00029189 CPU ] filename=snapshot-20-8260687-512469866 [ OK ] [ 0.02356348 / 0.00025057 CPU ] filename=snapshot-88-8260687-512469866.meta [ OK ] [ 0.02067455 / 0.00039195 CPU ] filename=snapshot-88-8260999-512469866.meta [ OK ] [ 0.01865136 / 0.00042170 CPU ] filename=tmp-snapshot-88-8260999-512469866.meta [ OK ] [ 0.01231359 / 0.00049213 CPU ] filename=tmp-snapshot-33-8260687-512469866 [ OK ] [ 0.02143644 / 0.00044466 CPU ] filename=snapshot-33-8260687-512469866.meta [ OK ] [ 0.01946592 / 0.00045430 CPU ] filename=tmp-metadata1 [ OK ] [ 0.01524396 / 0.00046636 CPU ] filename=tmp-metadata2 [ OK ] [ 0.01560362 / 0.00042354 CPU ] filename=tmp-open1 [ OK ] [ 0.01562170 / 0.00042841 CPU ] filename=tmp-open13 [ OK ] [ 0.01650461 / 0.00042856 CPU ] load/ignoreUnknownFiles filename=garbage [ OK ] [ 0.02549974 / 0.00040024 CPU ] filename=0000000000000000000000000001-00000000001garbage [ OK ] [ 0.01653129 / 0.00040585 CPU ] filename=open-1garbage [ OK ] [ 0.02011413 / 0.00056963 CPU ] load/emptyDir [ OK ] [ 0.01612761 / 0.00049136 CPU ] bootstrap/termIsNonZero [ OK ] [ 0.00023185 / 0.00023004 CPU ] bootstrap/pristine [ OK ] [ 0.00012437 / 0.00012251 CPU ] append/blockingBarrierLongBlockingTask [ OK ] [ 1.00089391 / 0.00158866 CPU ] append/nonBlockingBarrierLongBlockingTask [ OK ] [ 1.00095532 / 0.00181573 CPU ] append/blockingBarrierSingleOpenSegment [ OK ] [ 0.05094715 / 0.00373029 CPU ] append/blockingBarrierNoOpenSegments [ OK ] [ 0.03723240 / 0.00236615 CPU ] append/twoBarriersExitEarly bool=0 [ OK ] [ 0.00087394 / 0.00128152 CPU ] bool=1 [ OK ] [ 0.00089838 / 0.00133737 CPU ] append/twoBarriersOpenSegments bool=0 [ OK ] [ 0.04801691 / 0.00254564 CPU ] bool=1 [ OK ] [ 0.04404380 / 0.00378785 CPU ] append/barrierOpenSegmentsExitEarly bool=0 [ OK ] [ 0.00085493 / 0.00122802 CPU ] bool=1 [ OK ] [ 0.00096430 / 0.00147051 CPU ] append/barrierOpenSegments bool=0 [ OK ] [ 0.04901658 / 0.00286451 CPU ] bool=1 [ OK ] [ 0.05104781 / 0.00292950 CPU ] append/ioSetupError [ OK ] [ 0.00280499 / 0.00306386 CPU ] append/currentSegment [ OK ] [ 0.02604487 / 0.00168002 CPU ] append/cancelPrepareRequest [ SKIP ] append/removeSegmentUponClose [ OK ] [ 0.02192194 / 0.00163581 CPU ] append/closeDuringWrite [ SKIP ] append/oom heap-fault-delay=1, heap-fault-repeat=1 [ OK ] [ 0.00005244 / 0.00005138 CPU ] append/writeError [ SKIP ] append/noSpaceResolved dir-fs=tmpfs [ SKIP ] append/noSpaceUponWrite dir-fs=tmpfs [ SKIP ] append/noSpaceUponPrepareSpare dir-fs=tmpfs [ SKIP ] append/noSpaceUponPrepareCurrent dir-fs=tmpfs [ SKIP ] append/cancel [ OK ] [ 0.00058578 / 0.00091136 CPU ] append/counter [ OK ] [ 0.05093786 / 0.00339481 CPU ] append/prepareClosing [ OK ] [ 0.02603025 / 0.00165898 CPU ] append/truncateClosing [ OK ] [ 0.01792100 / 0.00156811 CPU ] append/truncate [ SKIP ] append/resizeArena [ OK ] [ 0.04541712 / 0.00246979 CPU ] append/wait [ OK ] [ 0.00091820 / 0.00138206 CPU ] append/batch [ OK ] [ 0.00081844 / 0.00127186 CPU ] append/exceedBlock [ OK ] [ 0.03440956 / 0.00226049 CPU ] append/matchBlock [ OK ] [ 0.03335376 / 0.00232952 CPU ] append/fitBlock [ OK ] [ 0.03822605 / 0.00220232 CPU ] append/severalBig [ OK ] [ 0.07285379 / 0.00485751 CPU ] append/secondBig [ OK ] [ 0.01937187 / 0.00193942 CPU ] append/firstBig [ OK ] [ 0.03341113 / 0.00267628 CPU ] append/finalizeSegment [ OK ] [ 0.01957565 / 0.00227292 CPU ] append/prepareSegments [ OK ] [ 0.00107050 / 0.00149898 CPU ] append/first [ OK ] [ 0.03534458 / 0.00244485 CPU ] append/unaligned [ OK ] [ 0.00004774 / 0.00004568 CPU ] init/metadataOneAndTwoSameVersion [ OK ] [ 0.01356537 / 0.00037486 CPU ] init/metadataOneBadVersion [ OK ] [ 0.02311655 / 0.00035764 CPU ] init/metadataOneBadFormat [ OK ] [ 0.02441296 / 0.00022390 CPU ] init/metadataOneTooShort [ OK ] [ 0.02056426 / 0.00026399 CPU ] init/noSpace dir-fs=tmpfs [ SKIP ] init/dirNotAccessible [ OK ] [ 0.00004819 / 0.00004760 CPU ] init/dirDoesNotExist [ OK ] [ 0.00004786 / 0.00004731 CPU ] init/probeAsyncIoOom [ SKIP ] init/probeDirectIoOom [ SKIP ] init/dirTooLong [ OK ] [ 0.00001749 / 0.00001675 CPU ] 201 of 227 (89%) tests successful, 13 (5%) test skipped. FAIL test/integration/uv (exit status: 1) ============================================================================ Testsuite summary for raft 0.18.3 ============================================================================ # TOTAL: 5 # PASS: 1 # SKIP: 0 # XFAIL: 0 # FAIL: 4 # XPASS: 0 # ERROR: 0 ============================================================================ See ./test-suite.log ============================================================================ ```
freeekanayaka commented 10 months ago

Can you provide the following details please?

kloczek commented 10 months ago
+ ./configure --bindir=/usr/bin --build=x86_64-redhat-linux --datadir=/usr/share --disable-dependency-tracking --exec-prefix=/usr --host=x86_64-redhat-linux --includedir=/usr/include --infodir=/usr/share/info --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --mandir=/usr/share/man --prefix=/usr --program-prefix= --runstatedir=/run --sbindir=/usr/sbin --sharedstatedir=/var/lib --sysconfdir=/etc --disable-static --enable-benchmark
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking whether make supports the include directive... yes (GNU style)
checking for x86_64-redhat-linux-gcc... /usr/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether /usr/bin/gcc accepts -g... yes
checking for /usr/bin/gcc option to enable C11 features... none needed
checking whether /usr/bin/gcc understands -c and -o together... yes
checking dependency style of /usr/bin/gcc... none
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for wchar.h... yes
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking whether _XOPEN_SOURCE should be defined... no
checking build system type... x86_64-redhat-linux-gnu
checking host system type... x86_64-redhat-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by /usr/bin/gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/gcc-nm
checking the name lister (/usr/bin/gcc-nm) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-redhat-linux-gnu file names to x86_64-redhat-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-redhat-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for x86_64-redhat-linux-file... no
checking for file... file
checking for x86_64-redhat-linux-objdump... no
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for x86_64-redhat-linux-dlltool... no
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for x86_64-redhat-linux-ar... /usr/bin/gcc-ar
checking for archiver @FILE support... @
checking for x86_64-redhat-linux-strip... no
checking for strip... strip
checking for x86_64-redhat-linux-ranlib... /usr/bin/gcc-ranlib
checking command to parse /usr/bin/gcc-nm output from /usr/bin/gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for x86_64-redhat-linux-mt... no
checking for mt... no
checking if : is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if /usr/bin/gcc supports -fno-rtti -fno-exceptions... no
checking for /usr/bin/gcc option to produce PIC... -fPIC -DPIC
checking if /usr/bin/gcc PIC flag -fPIC -DPIC works... yes
checking if /usr/bin/gcc static flag -static works... no
checking if /usr/bin/gcc supports -c -o file.o... yes
checking if /usr/bin/gcc supports -c -o file.o... (cached) yes
checking whether the /usr/bin/gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking for x86_64-redhat-linux-pkg-config... no
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for UV... yes
checking for LZ4... yes
checking whether to build with code coverage support... no
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for stdio.h... (cached) yes
checking for assert.h... yes
checking for unistd.h... (cached) yes
checking for linux/io_uring.h... yes
checking for linux/aio_abi.h... yes
checking for /usr/bin/gcc options needed to detect all undeclared functions... none needed
checking whether RWF_NOWAIT is declared... yes
checking for zfs... no
checking for size_t... yes
checking for ssize_t... yes
checking for uint8_t... yes
checking for uint16_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for /usr/bin/gcc option to enable large file support... none needed
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating raft.pc
config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands

[tkloczko@pers-jacek .build_logs]$ lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         46 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  48
  On-line CPU(s) list:   0-47
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Xeon(R) Silver 4116 CPU @ 2.10GHz
    CPU family:          6
    Model:               85
    Thread(s) per core:  2
    Core(s) per socket:  12
    Socket(s):           2
    Stepping:            4
    CPU(s) scaling MHz:  28%
    CPU max MHz:         3000.0000
    CPU min MHz:         800.0000
    BogoMIPS:            4200.00
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc ar
                         t arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid
                         dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 pti intel_ppin ssbd mba i
                         brs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap c
                         lflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts vnmi pku
                          ospke md_clear flush_l1d
Virtualization features:
  Virtualization:        VT-x
Caches (sum of all):
  L1d:                   768 KiB (24 instances)
  L1i:                   768 KiB (24 instances)
  L2:                    24 MiB (24 instances)
  L3:                    33 MiB (2 instances)
NUMA:
  NUMA node(s):          2
  NUMA node0 CPU(s):     0-11,24-35
  NUMA node1 CPU(s):     12-23,36-47
Vulnerabilities:
  Gather data sampling:  Vulnerable: No microcode
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Mitigation; Clear CPU buffers; SMT vulnerable
  Retbleed:              Mitigation; IBRS
  Spec rstack overflow:  Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Not affected
  Tsx async abort:       Mitigation; Clear CPU buffers; SMT vulnerable

freeekanayaka commented 10 months ago

Could you apply this patch:

diff --git a/src/configuration.c b/src/configuration.c
index c7782f78..1da3f5ab 100644
--- a/src/configuration.c
+++ b/src/configuration.c
@@ -330,13 +330,16 @@ int configurationDecode(const struct raft_buffer *buf,
     cursor = buf->base;

     /* Check the encoding format version */
-    if (byteGet8(&cursor) != ENCODING_FORMAT) {
+    uint8_t format = byteGet8(&cursor);
+    fprintf(stderr, "ENCODING FORMAT %d\n", format);
+    if (format != ENCODING_FORMAT) {
         rv = RAFT_MALFORMED;
         goto err;
     }

     /* Read the number of servers. */
     n = (size_t)byteGet64(&cursor);
+    fprintf(stderr, "N %zu\n", n);

     /* Decode the individual servers. */
     for (i = 0; i < n; i++) {
@@ -346,11 +349,13 @@ int configurationDecode(const struct raft_buffer *buf,

         /* Server ID. */
         id = byteGet64(&cursor);
+        fprintf(stderr, "ID %llu\n", id);

         /* Server Address. */
         address = byteGetString(
             &cursor,
             buf->len - (size_t)((uint8_t *)cursor - (uint8_t *)buf->base));
+        fprintf(stderr, "ADDRESS %p\n", (void *)address);
         if (address == NULL) {
             rv = RAFT_MALFORMED;
             goto err;
@@ -360,6 +365,7 @@ int configurationDecode(const struct raft_buffer *buf,
         role = byteGet8(&cursor);

         rv = configurationAdd(c, id, address, role);
+        fprintf(stderr, "ADD %d\n", rv);
         if (rv != 0) {
             /* Only valid configurations should be ever be encoded, so in case
              * configurationAdd() fails because of invalid data we return

and then run:

make test/unit/core && ./test/unit/core configurationDecode/oom --param heap-fault-delay 0 --show-stderr

?

kloczek commented 10 months ago

Here is the result:

[tkloczko@pers-jacek raft-0.18.3]$ make test/unit/core && ./test/unit/core configurationDecode/oom --param heap-fault-delay 0 --show-stderr
make: 'test/unit/core' is up to date.
Running test suite with seed 0x7db5b784...
configurationDecode/oom
  heap-fault-delay=0, heap-fault-repeat=1                   [ ERROR ]
ENCODING FORMAT 1
N 2
ID 5
ADDRESS 0x7fff67b89231
ADD 5
Error: test/unit/test_configuration.c:588: assertion failed: configurationDecode(&buf, &f->configuration) == RAFT_NOMEM (6 == 1)
Error: child killed by signal 6 (Aborted)
0 of 1 (0%) tests successful, 0 (0%) test skipped.
freeekanayaka commented 10 months ago

Thanks, could you reset the changes from the previous patch, and apply this new patch instead?

diff --git a/src/configuration.c b/src/configuration.c
index c7782f78..f6ef45bd 100644
--- a/src/configuration.c
+++ b/src/configuration.c
@@ -330,13 +330,16 @@ int configurationDecode(const struct raft_buffer *buf,
     cursor = buf->base;

     /* Check the encoding format version */
-    if (byteGet8(&cursor) != ENCODING_FORMAT) {
+    uint8_t format = byteGet8(&cursor);
+    fprintf(stderr, "ENCODING FORMAT %d\n", format);
+    if (format != ENCODING_FORMAT) {
         rv = RAFT_MALFORMED;
         goto err;
     }

     /* Read the number of servers. */
     n = (size_t)byteGet64(&cursor);
+    fprintf(stderr, "N %zu\n", n);

     /* Decode the individual servers. */
     for (i = 0; i < n; i++) {
@@ -346,11 +349,17 @@ int configurationDecode(const struct raft_buffer *buf,

         /* Server ID. */
         id = byteGet64(&cursor);
+        fprintf(stderr, "ID %llu\n", id);

         /* Server Address. */
         address = byteGetString(
             &cursor,
             buf->len - (size_t)((uint8_t *)cursor - (uint8_t *)buf->base));
+        if (address != NULL) {
+            fprintf(stderr, "ADDRESS %s LEN %zu\n", address, strlen(address));
+        } else {
+            fprintf(stderr, "NULL ADDRESS\n");
+        }
         if (address == NULL) {
             rv = RAFT_MALFORMED;
             goto err;
@@ -358,8 +367,10 @@ int configurationDecode(const struct raft_buffer *buf,

         /* Role code. */
         role = byteGet8(&cursor);
+        fprintf(stderr, "ROLE %d\n", role);

         rv = configurationAdd(c, id, address, role);
+        fprintf(stderr, "ADD %d\n", rv);
         if (rv != 0) {
             /* Only valid configurations should be ever be encoded, so in case
              * configurationAdd() fails because of invalid data we return

Then again run:

make test/unit/core && ./test/unit/core configurationDecode/oom --param heap-fault-delay 0 --show-stderr
kloczek commented 10 months ago
result ```console [tkloczko@pers-jacek raft-0.18.3]$ make test/unit/core && ./test/unit/core configurationDecode/oom --param heap-fault-delay 0 --show-stderr make: 'test/unit/core' is up to date. Running test suite with seed 0xcb375716... configurationDecode/oom heap-fault-delay=0, heap-fault-repeat=1 [ ERROR ] ENCODING FORMAT 1 N 2 ID 5 ADDRESS x.y LEN 3 ROLE 120 ADD 5 Error: test/unit/test_configuration.c:588: assertion failed: configurationDecode(&buf, &f->configuration) == RAFT_NOMEM (6 == 1) Error: child killed by signal 6 (Aborted) 0 of 1 (0%) tests successful, 0 (0%) test skipped. ```
freeekanayaka commented 10 months ago

The value of "ROLE" is wrong, it should be 1, not 120. Not sure what's going on.

New patch to apply and then re-run the test please:

diff --git a/src/configuration.c b/src/configuration.c
index c7782f78..6c4377a8 100644
--- a/src/configuration.c
+++ b/src/configuration.c
@@ -330,13 +330,16 @@ int configurationDecode(const struct raft_buffer *buf,
     cursor = buf->base;

     /* Check the encoding format version */
-    if (byteGet8(&cursor) != ENCODING_FORMAT) {
+    uint8_t format = byteGet8(&cursor);
+    fprintf(stderr, "ENCODING FORMAT %d\n", format);
+    if (format != ENCODING_FORMAT) {
         rv = RAFT_MALFORMED;
         goto err;
     }

     /* Read the number of servers. */
     n = (size_t)byteGet64(&cursor);
+    fprintf(stderr, "N %zu\n", n);

     /* Decode the individual servers. */
     for (i = 0; i < n; i++) {
@@ -346,11 +349,22 @@ int configurationDecode(const struct raft_buffer *buf,

         /* Server ID. */
         id = byteGet64(&cursor);
+        fprintf(stderr, "ID %llu\n", id);

         /* Server Address. */
+        for (int j = 0; j < 17; j++) {
+            fprintf(stderr, "%u ", *((uint8_t *)(cursor) + j));
+        }
+        fprintf(stderr, "\n");
         address = byteGetString(
             &cursor,
             buf->len - (size_t)((uint8_t *)cursor - (uint8_t *)buf->base));
+        fprintf(stderr, "ADDRESS %p\n", (void *)address);
+        if (address != NULL) {
+            fprintf(stderr, "ADDRESS %s LEN %zu\n", address, strlen(address));
+        } else {
+            fprintf(stderr, "NULL ADDRESS\n");
+        }
         if (address == NULL) {
             rv = RAFT_MALFORMED;
             goto err;
@@ -358,8 +372,10 @@ int configurationDecode(const struct raft_buffer *buf,

         /* Role code. */
         role = byteGet8(&cursor);
+        fprintf(stderr, "ROLE %d\n", role);

         rv = configurationAdd(c, id, address, role);
+        fprintf(stderr, "ADD %d\n", rv);
         if (rv != 0) {
             /* Only valid configurations should be ever be encoded, so in case
              * configurationAdd() fails because of invalid data we return
kloczek commented 10 months ago

result

[tkloczko@pers-jacek raft-0.18.3]$ make test/unit/core && ./test/unit/core configurationDecode/oom --param heap-fault-delay 0 --show-stderr
make: 'test/unit/core' is up to date.
Running test suite with seed 0x5ceace55...
configurationDecode/oom
  heap-fault-delay=0, heap-fault-repeat=1                   [ ERROR ]
ENCODING FORMAT 1
N 2
ID 5
120 46 121 0 1 3 0 0 0 0 0 0 0 122 46 119 0
ADDRESS 0x7ffddb024091
ADDRESS x.y LEN 3
ROLE 120
ADD 5
Error: test/unit/test_configuration.c:588: assertion failed: configurationDecode(&buf, &f->configuration) == RAFT_NOMEM (6 == 1)
Error: child killed by signal 6 (Aborted)
0 of 1 (0%) tests successful, 0 (0%) test skipped.
freeekanayaka commented 10 months ago
diff --git a/src/configuration.c b/src/configuration.c
index c7782f78..515ed14e 100644
--- a/src/configuration.c
+++ b/src/configuration.c
@@ -330,13 +330,16 @@ int configurationDecode(const struct raft_buffer *buf,
     cursor = buf->base;

     /* Check the encoding format version */
-    if (byteGet8(&cursor) != ENCODING_FORMAT) {
+    uint8_t format = byteGet8(&cursor);
+    fprintf(stderr, "ENCODING FORMAT %d\n", format);
+    if (format != ENCODING_FORMAT) {
         rv = RAFT_MALFORMED;
         goto err;
     }

     /* Read the number of servers. */
     n = (size_t)byteGet64(&cursor);
+    fprintf(stderr, "N %zu\n", n);

     /* Decode the individual servers. */
     for (i = 0; i < n; i++) {
@@ -346,11 +349,24 @@ int configurationDecode(const struct raft_buffer *buf,

         /* Server ID. */
         id = byteGet64(&cursor);
+        fprintf(stderr, "ID %llu\n", id);

         /* Server Address. */
+        fprintf(stderr, "CURSOR %p (%d)\n", cursor, *((uint8_t *)cursor));
+        for (int j = 0; j < 17; j++) {
+            fprintf(stderr, "%u ", *((uint8_t *)(cursor) + j));
+        }
+        fprintf(stderr, "\n");
         address = byteGetString(
             &cursor,
             buf->len - (size_t)((uint8_t *)cursor - (uint8_t *)buf->base));
+        fprintf(stderr, "ADDRESS %p\n", (void *)address);
+        fprintf(stderr, "CURSOR %p (%d)\n", cursor, *((uint8_t *)cursor));
+        if (address != NULL) {
+            fprintf(stderr, "ADDRESS %s LEN %zu\n", address, strlen(address));
+        } else {
+            fprintf(stderr, "NULL ADDRESS\n");
+        }
         if (address == NULL) {
             rv = RAFT_MALFORMED;
             goto err;
@@ -358,8 +374,11 @@ int configurationDecode(const struct raft_buffer *buf,

         /* Role code. */
         role = byteGet8(&cursor);
+        fprintf(stderr, "ROLE %d\n", role);
+        fprintf(stderr, "CURSOR %p (%d)\n", cursor, *((uint8_t *)cursor));

         rv = configurationAdd(c, id, address, role);
+        fprintf(stderr, "ADD %d\n", rv);
         if (rv != 0) {
             /* Only valid configurations should be ever be encoded, so in case
              * configurationAdd() fails because of invalid data we return
kloczek commented 10 months ago

result

[tkloczko@pers-jacek raft-0.18.3]$ make test/unit/core && ./test/unit/core configurationDecode/oom --param heap-fault-delay 0 --show-stderr                                                  make: 'test/unit/core' is up to date.
Running test suite with seed 0xbe756c11...
configurationDecode/oom
  heap-fault-delay=0, heap-fault-repeat=1                   [ OK    ] [ 0.00012779 / 0.00012652 CPU ]
ENCODING FORMAT 1
N 2
ID 5
CURSOR 0x7fffe0ae38a1 (120)
120 46 121 0 1 3 0 0 0 0 0 0 0 122 46 119 0
ADDRESS 0x7fffe0ae38a1
CURSOR 0x7fffe0ae38a5 (1)
ADDRESS x.y LEN 3
ROLE 1
CURSOR 0x7fffe0ae38a6 (3)
ADD 1
1 of 1 (100%) tests successful, 0 (0%) test skipped.
freeekanayaka commented 10 months ago

Ok, that's weird, simply printing the pointer makes the failure go away. I suspect there's some compilation flag that might create the problem.

Can you please attach here the content of these files on your system:

cat /usr/lib/rpm/redhat/redhat-hardened-cc1
cat /usr/lib/rpm/redhat/redhat-annobin-cc1

?

kloczek commented 10 months ago
[tkloczko@pers-jacek SPECS]$ cat /usr/lib/rpm/redhat/redhat-hardened-cc1
*cc1_options:
+ %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}}}}}}

*cpp_options:
+ %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}}}}}}
[tkloczko@pers-jacek SPECS]$ cat /usr/lib/rpm/redhat/redhat-annobin-cc1
*cc1_options:
+ %{!-fno-use-annobin:%{!iplugindir*:%:find-plugindir()} -fplugin=annobin}

First one forces to compile all code by default as PIE. Second one -fplugin=annobin (which add additional ELF section with some data).

freeekanayaka commented 10 months ago

Ok, I figured it out.

For some reason your build process is not passing -fno-strict-aliasing to GCC, although that flag is explicitly listed in configure.ac, see here.

Please can you try:

make clean
make check CFLAGS="-fno-strict-aliasing"

?

freeekanayaka commented 10 months ago

PS: run the above make targets against a clean raft v0.18.3 source tree, without any patch applied.

kloczek commented 10 months ago

I just realized that In patch which I'm using I've removed -fno-strict-aliasing. Original intention of that patch was to remove all hardcoded warning and optimisation flags to control that options from outside of the build procedure describes in rpm spec file. I need that kind of conditions to be able to perform additional test builds in my build infra. If your code relies on not use aliasing that is oddest thing which I've ever seen. On scale of almost 5k packages which I have in my distro it would be FIRST such package ever.

kloczek commented 10 months ago

OK I've tested and with removed that patch and test suite passes.

============================================================================
Testsuite summary for raft 0.18.3
============================================================================
# TOTAL: 5
# PASS:  5
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[2]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/raft-0.18.3'

Q: why your code relies on exact optimisation options? 🤔

freeekanayaka commented 10 months ago

I think this is accidental, the code should not rely on -fno-strict-aliasing. I'll see if I can fix it.

kloczek commented 10 months ago

If you need some set of warning flags please hardcode that options in your CI by use:

CFLAGS="<all_-W-foo_options_you_need>" \
./configure <ac_options>
make

or at least please provide autoconf option to allow drop all those options to allow generate

In some control builds which I;m using I'm building with some CC=<C_code_sanitiser>. I've not passed raft over all those control builds but some of the hardcoded warning flags will be causing such build to fail because some of those hardcoded options would b not recognised.

In am/ac files you are hardcoding warning it three ways.

All that makes really hard and/or impossible (especially with automake harcoding) to control those options from one place.

Q: in this case really on aliasing may affect installable ELF binaries or only test suite? 🤔

kloczek commented 10 months ago
Here is patch which I'm using ```patch --- a/configure.ac +++ b/configure.ac @@ -4,7 +4,7 @@ AC_CONFIG_MACRO_DIR([m4]) AC_CONFIG_AUX_DIR([ac]) -AM_INIT_AUTOMAKE([subdir-objects -Wall -Wno-portability foreign]) +AM_INIT_AUTOMAKE([subdir-objects foreign]) AM_SILENT_RULES([yes]) AC_USE_SYSTEM_EXTENSIONS # Defines _GNU_SOURCE and similar @@ -94,60 +94,6 @@ # on 32-bit architecture be 8 bytes instead of the normal 4. AC_SYS_LARGEFILE -CC_CHECK_FLAGS_APPEND([AM_CFLAGS],[CFLAGS],[ \ - -std=c11 \ - -g \ - --mcet \ - -fcf-protection \ - --param=ssp-buffer-size=4 \ - -pipe \ - -fno-strict-aliasing \ - -fdiagnostics-color \ - -fexceptions \ - -fstack-clash-protection \ - -fstack-protector-strong \ - -fasynchronous-unwind-tables \ - -fdiagnostics-show-option \ - -Wall \ - -Wextra \ - -Wpedantic \ - -Wimplicit-fallthrough=5 \ - -Wcast-align \ - -Wstrict-prototypes \ - -Wlogical-op \ - -Wmissing-include-dirs \ - -Wold-style-definition \ - -Winit-self \ - -Wfloat-equal \ - -Wsuggest-attribute=noreturn \ - -Wformat=2 \ - -Wendif-labels \ - -Wdate-time \ - -Wnested-externs \ - -Wconversion \ - -Werror=implicit-function-declaration \ - -Wunused-but-set-variable \ - -Werror=return-type \ - -Werror=incompatible-pointer-types \ - -Wshadow \ - -Werror=overflow \ - -Werror=shift-count-overflow \ - -Werror=shift-overflow=2 \ - -Warray-bounds \ - -Wrestrict \ - -Wreturn-local-addr \ - -Wstringop-overflow \ -]) -AC_SUBST(AM_CFLAGS) - -CC_CHECK_FLAGS_APPEND([AM_LDFLAGS],[LDFLAGS],[ \ - -z relro \ - -z now \ - -fstack-protector-strong \ - --param=ssp-buffer-size=4 \ -]) -AC_SUBST(AM_LDLAGS) - REQUIRES='' AS_IF(test "$have_uv" = yes, REQUIRES="$REQUIRES libuv") AS_IF(test "$have_lz4" = yes, REQUIRES="$REQUIRES liblz4") --- a/Makefile.am +++ b/Makefile.am @@ -1,8 +1,4 @@ ACLOCAL_AMFLAGS = -I m4 -AM_CFLAGS += $(CODE_COVERAGE_CFLAGS) - -# Not clobbering the base pointer helps bpftrace construct backtraces -AM_CFLAGS += -fno-omit-frame-pointer include_HEADERS = include/raft.h @@ -10,7 +6,7 @@ raftinclude_HEADERS = lib_LTLIBRARIES = libraft.la -libraft_la_CFLAGS = $(AM_CFLAGS) -fvisibility=hidden +libraft_la_CFLAGS = -fvisibility=hidden libraft_la_LDFLAGS = -version-info 0:0:0 libraft_la_SOURCES = \ src/byte.c \ @@ -54,7 +50,7 @@ check_LTLIBRARIES = libtest.la -libtest_la_CFLAGS = $(AM_CFLAGS) -DMUNIT_TEST_NAME_LEN=60 -Wno-unused-result -Wno-conversion +libtest_la_CFLAGS = -DMUNIT_TEST_NAME_LEN=60 libtest_la_SOURCES = \ test/lib/addrinfo.c \ test/lib/fault.c \ @@ -82,11 +78,10 @@ test/unit/test_log.c \ test/unit/test_queue.c \ test/unit/test_random.c -test_unit_core_CFLAGS = $(AM_CFLAGS) -Wno-conversion test_unit_core_LDADD = libtest.la if LZ4_AVAILABLE -test_unit_core_CFLAGS += -DLZ4_AVAILABLE $(LZ4_CFLAGS) +test_unit_core_CFLAGS = -DLZ4_AVAILABLE $(LZ4_CFLAGS) test_unit_core_LDFLAGS = $(LZ4_LIBS) libraft_la_CFLAGS += -DLZ4_AVAILABLE $(LZ4_CFLAGS) libraft_la_LDFLAGS += $(LZ4_LIBS) @@ -123,7 +118,6 @@ test/integration/test_strerror.c \ test/integration/test_tick.c \ test/integration/test_transfer.c -test_integration_core_CFLAGS = $(AM_CFLAGS) -Wno-conversion test_integration_core_LDFLAGS = -no-install test_integration_core_LDADD = libtest.la libraft.la @@ -133,7 +127,6 @@ test/fuzzy/test_liveness.c \ test/fuzzy/test_membership.c \ test/fuzzy/test_replication.c -test_fuzzy_core_CFLAGS = $(AM_CFLAGS) -Wno-conversion test_fuzzy_core_LDFLAGS = -no-install test_fuzzy_core_LDADD = libtest.la libraft.la @@ -187,7 +180,6 @@ test/unit/test_uv_fs.c \ test/unit/test_uv_os.c \ test/unit/test_uv_writer.c -test_unit_uv_CFLAGS = $(AM_CFLAGS) -Wno-conversion test_unit_uv_LDADD = libtest.la $(UV_LIBS) # The integration/uv test is not linked to libraft, but built @@ -209,14 +201,11 @@ test/integration/test_uv_snapshot_put.c \ test/integration/test_uv_truncate.c \ test/integration/test_uv_truncate_snapshot.c -test_integration_uv_CFLAGS = $(AM_CFLAGS) -Wno-type-limits -Wno-conversion test_integration_uv_LDFLAGS = -no-install test_integration_uv_LDADD = libtest.la $(UV_LIBS) -AM_CFLAGS += $(UV_CFLAGS) - if LZ4_AVAILABLE -test_integration_uv_CFLAGS += -DLZ4_AVAILABLE +test_integration_uv_CFLAGS = -DLZ4_AVAILABLE test_integration_uv_LDFLAGS += $(LZ4_LIBS) endif # LZ4_AVAILABLE @@ -262,16 +251,6 @@ endif # BENCHMARK_ENABLED -if DEBUG_ENABLED - AM_CFLAGS += -Werror -Wall -else - AM_CFLAGS += -DNDEBUG -endif - -if SANITIZE_ENABLED - AM_CFLAGS += -fsanitize=address -endif - if CODE_COVERAGE_ENABLED include $(top_srcdir)/aminclude_static.am ```

I you want to make your ac/am cleaner it needs to be done in few small steps to be able easy confirm that nothing is affected. In my patch I've ignored DEBUG and SANITIZER parts as not relevant for my build. If you want I can submit for that part part PR to move DEBUG that to only configire.ac. Than the same way could be simplified --enable-sanitize part.

kloczek commented 10 months ago

My patch looks like exposed places affected by aliasing

[tkloczko@pers-jacek SPECS]$ rpmbuild -ba raft.spec --quiet 2>&1 | grep -- \\[-W | sed 's/.*\[//; s/\]//' | sort | uniq -c | sort -nr
      4 -Wstrict-aliasing

Extracted part from stderr:

src/uv_fs.c: In function 'probeAsyncIO':
src/uv_fs.c:732:7: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  732 |     *((void **)(&iocb.aio_buf)) = buf;
      |      ~^~~~~~~~~~~~~~~~~~~~~~~~~
src/uv_writer.c: In function 'uvWriterPollCb':
src/uv_writer.c:193:37: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  193 |         struct UvWriterReq *req = *((void **)&event->data);
      |                                    ~^~~~~~~~~~~~~~~~~~~~~~
src/uv_writer.c: In function 'UvWriterSubmit':
src/uv_writer.c:475:7: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  475 |     *((void **)(&req->iocb.aio_buf)) = (void *)bufs;
      |      ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/uv_writer.c:478:7: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  478 |     *((void **)(&req->iocb.aio_data)) = (void *)req;
      |      ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
freeekanayaka commented 10 months ago

If you need some set of warning flags please hardcode that options in your should hardcode that in CI by use

CFLAGS="<all_-W-foo_options_you_need>" \
./configure <ac_options>
make

or at least please provide autoconf option to allow drop all those options to allow generate

  • build log with minimal length
  • to allow alter warning flags from outside
  • I'm not sure are all those already hardcoded options will be possible use with CC=clang.

Note that these flags are not mandatory, just recommended: the CC_CHECK_FLAGS_APPEND macros checks if a certain flag is available in the compiler being used (the one set with CC), and if it's not available that flag will not be used.

All the currently recommended flags should be available with CC=clang, FWIW. But you should be able to use other compilers just fine (e.g. CC=tcc).

As I said, -fno-strict-aliasing is currently not only recommended, but necessary. That's a bug that I have to fixed.

In some control builds which I;m using I'm building with some CC=<C_code_sanitiser>. I've not passed raft over all those control builds but some of the hardcoded warning flags will be causing such build to fail because some of those hardcoded options would b not recognised.

In am/ac files you are hardcoding warning it three ways.

  • in ac AM_INIT_AUTOMAKE([subdir-objects -Wall -Wno-portability foreign])
  • in ac CC_CHECK_FLAGS_APPEND()
  • in am in some *_CFLAGS=.

All that makes really hard and/or impossible (especially with automake harcoding) to control those options from one place.

Q: in this case really on aliasing may affect installable ELF binaries or only test suite? 🤔

It will affect the final libraft binary, not just the test suite.

kloczek commented 10 months ago

Note that these flags are not mandatory, just recommended: the CC_CHECK_FLAGS_APPEND macros checks if a certain flag is available in the compiler being used (the one set with CC), and if it's not available that flag will not be used.

I understand however CC_CHECK_FLAGS_APPEND aclocal macro is not associated with ac --{enable|disable}-warnings option.

All the currently recommended flags should be available with CC=clang, FWIW. But you should be able to use other compilers just fine (e.g. CC=tcc).

Both gcc and clang are constantly evolving and some gcc options are not handled by clang and vice versa. (IMO) default variant of the builds organized with use GNI autotool, cmake or meson should provide at least option allowing drop all warning options.

freeekanayaka commented 10 months ago

Note that these flags are not mandatory, just recommended: the CC_CHECK_FLAGS_APPEND macros checks if a certain flag is available in the compiler being used (the one set with CC), and if it's not available that flag will not be used.

I understand however CC_CHECK_FLAGS_APPEND aclocal macro is not associated with ac --{enable|disable}-warnings option.

I can surely add an --{enable|disable}-warnings option or something like that. By default those flags would be checked and used if available, but with --disable-warnings you can turn of that behavior.

All the currently recommended flags should be available with CC=clang, FWIW. But you should be able to use other compilers just fine (e.g. CC=tcc).

Both gcc and clang are constantly evolving and some gcc options are not handled by clang and vice versa.

Sure, but that's not a problem. If an option is not available CC_CHECK_FLAGS_APPEND will report that it's not available, and it will not be used.

(IMO) default variant of the builds organized with use GNI autotool, cmake or meson should provide at least option allowing drop all warning options.

Sure. You should be able to build the code however you wish. The code should just provide sane defaults that you can override.

kloczek commented 10 months ago

It will affect the final libraft binary, not just the test suite.

I'll keep my patch in my build procwdure + line like below i raft.spec:

%global         build_cflags    %{build_cflags} -fno-strict-aliasing

to have in spec file visibility that at least for now raft code depends on aliasing option.

freeekanayaka commented 10 months ago

It will affect the final libraft binary, not just the test suite.

I'll keep my patch in my build procwdure + line like below i raft.spec:

%global         build_cflags    %{build_cflags} -fno-strict-aliasing

to have in spec file visibility that at least for now raft code depends on aliasing option.

Thanks. I'll update this issue once I get rid of that dependency.

kloczek commented 10 months ago

Thanks. I'll update this issue once I get rid of that dependency.

Sorry to not provide some details initially but .. [mode="Monty Python] Nobody expects the Spanish Inquisition.[/mode] 😋

freeekanayaka commented 10 months ago

I'll keep my patch in my build procwdure

Note that you should be able to disable any of the recommended flags without patching configure.ac, for example if you don't want -Wall you can set CFLAGS=-Wno-all. So you can already achieve that without --disable-warnings (but I'm fine to add that).

freeekanayaka commented 10 months ago

E.g. make CFLAGS=-Wno-all.

kloczek commented 10 months ago

Yes I know .. however it makes build logs unnecessarily longer and/or harder to process automatically 😢

freeekanayaka commented 10 months ago

This bug should be fixed by #127, which I just merged

Please could you check that you are now able to drop the -fno-strict-aliasing flag?

There is also a new --disable-hardening configuration flag that you can use to disable all the hardening compiler options, as you had requested.

eli-schwartz commented 10 months ago

If your code relies on not use aliasing that is oddest thing which I've ever seen. On scale of almost 5k packages which I have in my distro it would be FIRST such package ever.

I... admire your naive optimism. :D Lots of code, sadly, depends on this unsafe compiler detail. (Distros do not patch those flags, though...)

Also I am nearly positive you package the linux kernel, also cpython. (There are other famous packages with the issue, but that's enough for now.)

...

@freeekanayaka I figure that your -std is not really a "compile-time hardening option and should still be set unconditionally?

freeekanayaka commented 10 months ago

@freeekanayaka I figure that your -std is not really a "compile-time hardening option and should still be set unconditionally?

Good point. Fixed that in #129.