search

erthink / libmdbx

One of the fastest embeddable key-value ACID database without WAL. libmdbx surpasses the legendary LMDB in terms of reliability, features and performance.
https://erthink.github.io/libmdbx/
Other
1.16k stars 111 forks source link

v0.10.3 Assertion Failure on iOS #235

Closed youjinp closed 3 years ago

youjinp commented 3 years ago

Problem

Hi,

After updating from 10.1 to 10.3, I'm getting this failure on mdbx_env_open

Assertion failed: ((uintptr_t)ptr % expected_alignment == 0), function unaligned_poke_u64, file mdbx, line 199.

I've gotten the same issue on 10.2

What have I missed?

Thank you

Build

Built using the iOS-toolchain:

cmake ../$MDBX_TEMP -G Xcode -DCMAKE_TOOLCHAIN_FILE=../$IOS_CMAKE_TEMP/ios.toolchain.cmake -DPLATFORM=OS64COMBINED -DENABLE_BITCODE=ON

Config

/* This is CMake-template for libmdbx's config.h
 ******************************************************************************/

/* *INDENT-OFF* */
/* clang-format off */

#define LTO_ENABLED
/* #undef MDBX_USE_VALGRIND */
/* #undef ENABLE_GPROF */
/* #undef ENABLE_GCOV */
/* #undef ENABLE_ASAN */
/* #undef ENABLE_UBSAN */
#define MDBX_FORCE_ASSERTIONS 0

/* Common */
#define MDBX_TXN_CHECKOWNER 1
#define MDBX_ENV_CHECKPID_AUTO
#ifndef MDBX_ENV_CHECKPID_AUTO
#define MDBX_ENV_CHECKPID 0
#endif
#define MDBX_LOCKING_AUTO
#ifndef MDBX_LOCKING_AUTO
/* #undef MDBX_LOCKING */
#endif
#define MDBX_TRUST_RTC_AUTO
#ifndef MDBX_TRUST_RTC_AUTO
#define MDBX_TRUST_RTC 0
#endif
#define MDBX_DISABLE_PAGECHECKS 0

/* Windows */
#define MDBX_WITHOUT_MSVC_CRT 0

/* MacOS & iOS */
#define MDBX_OSX_SPEED_INSTEADOF_DURABILITY 0

/* POSIX */
#define MDBX_DISABLE_GNU_SOURCE 0
#define MDBX_USE_OFDLOCKS_AUTO
#ifndef MDBX_USE_OFDLOCKS_AUTO
#define MDBX_USE_OFDLOCKS 0
#endif

/* Build Info */
#ifndef MDBX_BUILD_TIMESTAMP
#define MDBX_BUILD_TIMESTAMP "2021-10-08T04:15:57Z"
#endif
#ifndef MDBX_BUILD_TARGET
#define MDBX_BUILD_TARGET "Darwin-iOS"
#endif
#ifndef MDBX_BUILD_TYPE
#define MDBX_BUILD_TYPE "<CONFIGURATION DEPENDENT>"
#endif
#ifndef MDBX_BUILD_COMPILER
#define MDBX_BUILD_COMPILER "Apple clang version 13.0.0 (clang-1300.0.29.3)"
#endif
#ifndef MDBX_BUILD_FLAGS
#define MDBX_BUILD_FLAGS " -fexceptions -fcxx-exceptions -frtti -fno-common -ggdb -Wno-unknown-pragmas -Wall -Wextra -Werror -flto=thin LIBMDBX_EXPORTS MDBX_BUILD_SHARED_LIBRARY=1 -ffast-math -fvisibility=hidden"
#endif
#define MDBX_BUILD_SOURCERY 6f6e3fb6f04216d45d83e5d1201a1451ab52acb1e2172f47a292d970fd32379f_v0_10_3_0_gbf603bd

/* *INDENT-ON* */
/* clang-format on */

Version

/* This is CMake-template for libmdbx's version.c
 ******************************************************************************/

#include "internals.h"

#if MDBX_VERSION_MAJOR != 0 ||                             \
    MDBX_VERSION_MINOR != 10
#error "API version mismatch! Had `git fetch --tags` done?"
#endif

static const char sourcery[] = MDBX_STRINGIFY(MDBX_BUILD_SOURCERY);

__dll_export
#ifdef __attribute_used__
    __attribute_used__
#elif defined(__GNUC__) || __has_attribute(__used__)
    __attribute__((__used__))
#endif
#ifdef __attribute_externally_visible__
        __attribute_externally_visible__
#elif (defined(__GNUC__) && !defined(__clang__)) ||                            \
    __has_attribute(__externally_visible__)
    __attribute__((__externally_visible__))
#endif
    const struct MDBX_version_info mdbx_version = {
        0,
        10,
        3,
        0,
        {"2021-08-27T22:47:12+03:00", "89059f5e31a893b0739257a1df03e3d9fcc6d74d", "bf603bdffc22c00d393b7614c1f11f3ebe4efb25",
         "v0.10.3-0-gbf603bd"},
        sourcery};

__dll_export
#ifdef __attribute_used__
    __attribute_used__
#elif defined(__GNUC__) || __has_attribute(__used__)
    __attribute__((__used__))
#endif
#ifdef __attribute_externally_visible__
        __attribute_externally_visible__
#elif (defined(__GNUC__) && !defined(__clang__)) ||                            \
    __has_attribute(__externally_visible__)
    __attribute__((__externally_visible__))
#endif
    const char *const mdbx_sourcery_anchor = sourcery;
erthink commented 3 years ago

@youjinp, I will try to reproduce. But, please, provide a complete stack trace.

erthink commented 3 years ago

Currently I haven't an iOS environment to reproduce. But I have not been able to reproduce the problem on OSX and FreeBSD. Therefore, I will be able to understand what is the matter and fix the problem only if you, or someone else, provide at least a full stack trace.

youjinp commented 3 years ago

My apologies, here's the stack trace:

* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
    frame #0: 0x0000000191750e68 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x000000019178343c libsystem_pthread.dylib`pthread_kill + 292
    frame #2: 0x00000001916cb460 libsystem_c.dylib`abort + 104
    frame #3: 0x00000001916ca8f4 libsystem_c.dylib`__assert_rtn + 292
    * frame #4: 0x0000000119f0de18 MDBXTests`mdbx_assert_fail(env=0x0000000000000000, msg="(uintptr_t)ptr % expected_alignment == 0", func="unaligned_poke_u64", line=199) at osal.c:246:5
    frame #5: 0x0000000119eede98 MDBXTests`mdbx_read_header [inlined] unaligned_poke_u64(expected_alignment=4, ptr=0x000000016fdfbe9a, v=1) at core.c:199:3
    frame #6: 0x0000000119eede54 MDBXTests`mdbx_read_header(env=0x000000011b0b5440, dest=0x000000016fdfbdf2, lck_exclusive=-1, mode_bits=438) at core.c:10355:3
    frame #7: 0x0000000119ee721c MDBXTests`mdbx_setup_dxb(env=0x000000011b0b5440, lck_rc=-1, mode_bits=438) at core.c:11434:13
    frame #8: 0x0000000119f05b48 MDBXTests`mdbx_env_open(env=0x000000011b0b5440, pathname="/Users/youjin/Library/Developer/Xcode/DerivedData/swift-mdbx-gcrkcmkxgesjrbbuhpwbjbfyjnka/Build/Products/Debug/MDBXTests.xctest/mdbx", flags=MDBX_ENV_DEFAULTS, mode=438) at core.c:12604:22
    frame #9: 0x0000000119ef8c28 MDBXTests`MDBX.init(path="file:///Users/youjin/Library/Developer/Xcode/DerivedData/swift-mdbx-gcrkcmkxgesjrbbuhpwbjbfyjnka/Build/Products/Debug/MDBXTests.xctest/mdbx/", flags=(rawValue = 0), mode=(rawValue = 436), self=(env = 0x000000011b0b5440)) at MDBX.swift:30:24
    frame #10: 0x0000000119ef8434 MDBXTests`MDBX.__allocating_init(path:flags:mode:) at MDBX.swift:0
    frame #11: 0x0000000119e5302c MDBXTests`MDBXTests.testT(self=0x000000010181e0a0) at MDBXTests.swift:24:23
    frame #12: 0x0000000119e538e8 MDBXTests`@objc MDBXTests.testT() at <compiler-generated>:0
    frame #13: 0x0000000191863864 CoreFoundation`__invoking___ + 148
    frame #14: 0x00000001918636e8 CoreFoundation`-[NSInvocation invoke] + 380
    frame #15: 0x000000010049fcd0 XCTestCore`+[XCTFailableInvocation invokeStandardConventionInvocation:completion:] + 76
    frame #16: 0x000000010049fc7c XCTestCore`__65+[XCTFailableInvocation invokeInvocation:lastObservedErrorIssue:]_block_invoke_3 + 28
    frame #17: 0x000000010049f4bc XCTestCore`__81+[XCTFailableInvocation invokeWithAsynchronousWait:lastObservedErrorIssue:block:]_block_invoke.11 + 112
    frame #18: 0x00000001004181b0 XCTestCore`+[XCTSwiftErrorObservation observeErrorsInBlock:] + 92
    frame #19: 0x000000010049f314 XCTestCore`+[XCTFailableInvocation invokeWithAsynchronousWait:lastObservedErrorIssue:block:] + 492
    frame #20: 0x000000010049f8f8 XCTestCore`+[XCTFailableInvocation invokeInvocation:lastObservedErrorIssue:] + 388
    frame #21: 0x0000000100485c74 XCTestCore`__24-[XCTestCase invokeTest]_block_invoke_2 + 104
    frame #22: 0x00000001004283c4 XCTestCore`-[XCTMemoryChecker _assertInvalidObjectsDeallocatedAfterScope:] + 88
    frame #23: 0x000000010048fd1c XCTestCore`-[XCTestCase assertInvalidObjectsDeallocatedAfterScope:] + 72
    frame #24: 0x0000000100485bc8 XCTestCore`__24-[XCTestCase invokeTest]_block_invoke.292 + 192
    frame #25: 0x00000001003f2eac XCTestCore`-[XCTestCase(XCTIssueHandling) _caughtUnhandledDeveloperExceptionPermittingControlFlowInterruptions:caughtInterruptionException:whileExecutingBlock:] + 180
    frame #26: 0x00000001004856f4 XCTestCore`-[XCTestCase invokeTest] + 844
    frame #27: 0x0000000100487080 XCTestCore`__26-[XCTestCase performTest:]_block_invoke_2 + 48
    frame #28: 0x00000001003f2eac XCTestCore`-[XCTestCase(XCTIssueHandling) _caughtUnhandledDeveloperExceptionPermittingControlFlowInterruptions:caughtInterruptionException:whileExecutingBlock:] + 180
    frame #29: 0x0000000100486fb4 XCTestCore`__26-[XCTestCase performTest:]_block_invoke.408 + 124
    frame #30: 0x000000010043e4f4 XCTestCore`+[XCTContext runInContextForTestCase:markAsReportingBase:block:] + 228
    frame #31: 0x00000001004867fc XCTestCore`-[XCTestCase performTest:] + 740
    frame #32: 0x00000001003be358 XCTestCore`-[XCTest runTest] + 64
    frame #33: 0x0000000100441728 XCTestCore`-[XCTestSuite runTestBasedOnRepetitionPolicy:testRun:] + 160
    frame #34: 0x0000000100441594 XCTestCore`__27-[XCTestSuite performTest:]_block_invoke + 216
    frame #35: 0x0000000100440f34 XCTestCore`__59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 48
    frame #36: 0x000000010043e4f4 XCTestCore`+[XCTContext runInContextForTestCase:markAsReportingBase:block:] + 228
    frame #37: 0x0000000100440ed0 XCTestCore`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 192
    frame #38: 0x000000010044120c XCTestCore`-[XCTestSuite performTest:] + 240
    frame #39: 0x00000001003be358 XCTestCore`-[XCTest runTest] + 64
    frame #40: 0x0000000100441728 XCTestCore`-[XCTestSuite runTestBasedOnRepetitionPolicy:testRun:] + 160
    frame #41: 0x0000000100441594 XCTestCore`__27-[XCTestSuite performTest:]_block_invoke + 216
    frame #42: 0x0000000100440f34 XCTestCore`__59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 48
    frame #43: 0x000000010043e4f4 XCTestCore`+[XCTContext runInContextForTestCase:markAsReportingBase:block:] + 228
    frame #44: 0x0000000100440ed0 XCTestCore`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 192
    frame #45: 0x000000010044120c XCTestCore`-[XCTestSuite performTest:] + 240
    frame #46: 0x00000001003be358 XCTestCore`-[XCTest runTest] + 64
    frame #47: 0x0000000100441728 XCTestCore`-[XCTestSuite runTestBasedOnRepetitionPolicy:testRun:] + 160
    frame #48: 0x0000000100441594 XCTestCore`__27-[XCTestSuite performTest:]_block_invoke + 216
    frame #49: 0x0000000100440f34 XCTestCore`__59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 48
    frame #50: 0x000000010043e4f4 XCTestCore`+[XCTContext runInContextForTestCase:markAsReportingBase:block:] + 228
    frame #51: 0x0000000100440ed0 XCTestCore`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 192
    frame #52: 0x000000010044120c XCTestCore`-[XCTestSuite performTest:] + 240
    frame #53: 0x00000001003be358 XCTestCore`-[XCTest runTest] + 64
    frame #54: 0x00000001003c04b4 XCTestCore`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke_2 + 160
    frame #55: 0x000000010043e4f4 XCTestCore`+[XCTContext runInContextForTestCase:markAsReportingBase:block:] + 228
    frame #56: 0x00000001003c0404 XCTestCore`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 152
    frame #57: 0x00000001003c05c8 XCTestCore`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.98 + 116
    frame #58: 0x00000001003eafc8 XCTestCore`-[XCTestObservationCenter _observeTestExecutionForBlock:] + 340
    frame #59: 0x00000001003c0154 XCTestCore`-[XCTTestRunSession runTestsAndReturnError:] + 504
    frame #60: 0x000000010043ba44 XCTestCore`-[XCTestDriver _runTests] + 464
    frame #61: 0x00000001003be9fc XCTestCore`_XCTestMain + 100
    frame #62: 0x00000001000063a8 xctest`main + 272
    frame #63: 0x00000001917a1450 libdyld.dylib`start + 4
erthink commented 3 years ago

Taking look to the frames 7 and 6 we see that the mdbx_setup_dxb(env=0x000000011b0b5440, lck_rc=-1, mode_bits=438) https://github.com/erthink/libmdbx/blob/bf603bdffc22c00d393b7614c1f11f3ebe4efb25/src/core.c#L11430-L11434 calls the mdbx_read_header(env=0x000000011b0b5440, dest=0x000000016fdfbdf2, lck_exclusive=-1, mode_bits=438) https://github.com/erthink/libmdbx/blob/bf603bdffc22c00d393b7614c1f11f3ebe4efb25/src/core.c#L10347-L10355

During this call the dest param of callee points to the local variable meta of the caller. The meta variable is placed on the stack frame and there is no reason for it to be unaligned. However, actually the dest pointer is 0x000000016fdfbdf2.

I have no explanation for this yet, except a compiler' bug.

erthink commented 3 years ago

@youjinp, as an experiment, suggest you trying to change the https://github.com/erthink/libmdbx/blob/bf603bdffc22c00d393b7614c1f11f3ebe4efb25/src/internals.h#L440 to #pragma pack(push, 4).

youjinp commented 3 years ago

Well.. that worked!

erthink commented 3 years ago

Well.. that worked!

Ok, I will commit a such fix. Thanks for reporting.