search

Tokutek / mongo

TokuMX is a high-performance, concurrent, compressing, drop-in replacement engine for MongoDB | Issue tracker: https://tokutek.atlassian.net/browse/MX/ |
http://www.tokutek.com/products/tokumx-for-mongodb/
703 stars 97 forks source link

Aborted when run #1229

Open fydot opened 9 years ago

fydot commented 9 years ago

when I run, I only got aborted

~$ /usr/bin/mongod --config /data/data/tokumx-cluster/shard2/tokumx.conf Mon Jun 8 22:41:47.280 warning: Mon Jun 8 22:41:47.280 warning: No value set for expireOplogDays, using default of 14 days. Mon Jun 8 22:41:47.280 warning: Mon Jun 8 22:41:47.280 nssize is a deprecated parameter all output going to: /data/data/tokumx-cluster/shard2/log/shard2.log Aborted

log is below:

* SERVER RESTARTED *

Mon Jun 8 22:41:47.285 [initandlisten] TokuMX starting : pid=20477 port=27002 dbpath=/data/data/tokumx-cluster/shard2/data 64-bit host=tokumx01 Mon Jun 8 22:41:47.285 [initandlisten] TokuMX mongod server v2.0.1-mongodb-2.4.10, using TokuKV rev 39fefcc4d2b1421ea09a9e41b130a64b230583b7 Mon Jun 8 22:41:47.285 [initandlisten] git version: d4547074dcc9129f2e13ed6b76fd78edb0a2828e Mon Jun 8 22:41:47.285 [initandlisten] build info: Linux 62691929ac33 3.11.0-26-generic #45-Ubuntu SMP Tue Jul 15 04:02:06 UTC 2014 x86_64 GNU/Linux BOOST_LIB_VERSION=1_49 Mon Jun 8 22:41:47.285 [initandlisten] options: { config: "/data/data/tokumx-cluster/shard2/tokumx.conf", dbpath: "/data/data/tokumx-cluster/shard2/data", logappend: "true", logpath: "/data/data/tokumx-cluster/shard2/log/shard2.log", pluginsDir: "/usr/lib/tokumx/plugins", port: 27002, replSet: "shard2", shardsvr: "true" } Mon Jun 8 22:41:47.285 [initandlisten] [tokumx] startup Mon Jun 8 22:41:47 2015 TokuFT recovery starting in env /data/data/tokumx-cluster/shard2/data Mon Jun 8 22:41:47 2015 TokuFT recovery failed -30975 Mon Jun 8 22:41:47.402 [initandlisten] Assertion: 16762:Automatic environment recovery failed. There may be data corruption. 0xc4ced3 0x880f0b 0x882075 0x87a600 0x87ad9f 0x87b5bd 0x7b0cca 0x7b13a8 0x79b89a 0x7f68963cfead 0x7adcc9 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x23) [0xc4ced3] /usr/bin/mongod(_ZN5mongo7storage21MsgAssertionExceptionC2EiRKSs+0x9b) [0x880f0b] /usr/bin/mongod(_ZN5mongo7storage23DataCorruptionExceptionC2EiRKSs+0xa5) [0x882075] /usr/bin/mongod(_ZN5mongo7storage16handle_ydb_errorEi+0x6e0) [0x87a600] /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0xf) [0x87ad9f] /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd) [0x87b5bd] /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a) [0x7b0cca] /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18) [0x7b13a8] /usr/bin/mongod(main+0x29a) [0x79b89a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f68963cfead] /usr/bin/mongod() [0x7adcc9] Mon Jun 8 22:41:47.408 [initandlisten] fatal error 16762: Automatic environment recovery failed. There may be data corruption. Mon Jun 8 22:41:47.408 [initandlisten] 16762 Automatic environment recovery failed. There may be data corruption. Mon Jun 8 22:41:47.408 [initandlisten] Fatal Assertion 16762 0xc4ced3 0xacd20c 0x87af86 0x87b5bd 0x7b0cca 0x7b13a8 0x79b89a 0x7f68963cfead 0x7adcc9 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x23) [0xc4ced3] /usr/bin/mongod(_ZN5mongo13fassertFailedEi+0x4c) [0xacd20c] /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0x1f6) [0x87af86] /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd) [0x87b5bd] /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a) [0x7b0cca] /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18) [0x7b13a8] /usr/bin/mongod(main+0x29a) [0x79b89a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f68963cfead] /usr/bin/mongod() [0x7adcc9] Mon Jun 8 22:41:47.412 [initandlisten]

***aborting after fassert() failure

Mon Jun 8 22:41:47.412 Mon Jun 8 22:41:47.412 ================================================================================ Mon Jun 8 22:41:47.412 Fatal error detected Mon Jun 8 22:41:47.412 ================================================================================ Mon Jun 8 22:41:47.412 Mon Jun 8 22:41:47.412 About to gather debugging information, please include all of the following along Mon Jun 8 22:41:47.412 with logs from other servers in the cluster in a bug report. Mon Jun 8 22:41:47.412 Mon Jun 8 22:41:47.412 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.412 Version info: Mon Jun 8 22:41:47.412 Mon Jun 8 22:41:47.412 tokumxVersion: 2.0.1 Mon Jun 8 22:41:47.412 gitVersion: d4547074dcc9129f2e13ed6b76fd78edb0a2828e Mon Jun 8 22:41:47.412 tokukvVersion: 39fefcc4d2b1421ea09a9e41b130a64b230583b7 Mon Jun 8 22:41:47.412 sysInfo: Linux 62691929ac33 3.11.0-26-generic #45-Ubuntu SMP Tue Jul 15 04:02:06 UTC 2014 x86_64 GNU/Linux BOOST_LIB_VERSION=1_49 Mon Jun 8 22:41:47.412 loaderFlags: Mon Jun 8 22:41:47.412 compilerFlags: -fPIC -fno-strict-aliasing -ggdb -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -pipe -Wnon-virtual-dtor -Woverloaded-virtual -Wno-unused-local-typedefs -fno-builtin-memcmp -O3 Mon Jun 8 22:41:47.412 debug: false Mon Jun 8 22:41:47.412 Mon Jun 8 22:41:47.412 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.412 Simple stacktrace: Mon Jun 8 22:41:47.412 0xc4ced3 0x989157 0x98c5eb 0x7ae1d7 0x7f68963e31e0 0x7f68963e3165 0x7f68963e63e0 0xacd247 0x87af86 0x87b5bd 0x7b0cca 0x7b13a8 0x79b89a 0x7f68963cfead 0x7adcc9 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x23) [0xc4ced3] /usr/bin/mongod() [0x989157] /usr/bin/mongod(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x2b) [0x98c5eb] /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0xa7) [0x7ae1d7] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f68963e31e0] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f68963e3165] /lib/x86_64-linux-gnu/libc.so.6(abort+0x180) [0x7f68963e63e0] /usr/bin/mongod(_ZN5mongo13fassertFailedEi+0x87) [0xacd247] /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0x1f6) [0x87af86] /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd) [0x87b5bd] /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a) [0x7b0cca] /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18) [0x7b13a8] /usr/bin/mongod(main+0x29a) [0x79b89a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f68963cfead] /usr/bin/mongod() [0x7adcc9] Mon Jun 8 22:41:47.417 Mon Jun 8 22:41:47.417 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.417 TokuKV engine backtrace: Mon Jun 8 22:41:47.417 Mon Jun 8 22:41:47.417 Backtrace: (Note: toku_do_assert=0x0x7f6897947d90)

Mon Jun 8 22:41:47.422 /usr/lib/tokumx/libtokuportability.so(_Z27db_env_do_backtraceerrfuncPFvPKviPKczES0+0x1f) [0x7f6897949a9f] Mon Jun 8 22:41:47.422 /usr/lib/tokumx/libtokufractaltree.so(+0x14bdd) [0x7f6897b63bdd] Mon Jun 8 22:41:47.422 /usr/bin/mongod() [0x989228] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x2b) [0x98c5eb] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0xa7) [0x7ae1d7] Mon Jun 8 22:41:47.422 /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f68963e31e0] Mon Jun 8 22:41:47.422 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f68963e3165] Mon Jun 8 22:41:47.422 /lib/x86_64-linux-gnu/libc.so.6(abort+0x180) [0x7f68963e63e0] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo13fassertFailedEi+0x87) [0xacd247] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0x1f6) [0x87af86] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd) [0x87b5bd] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a) [0x7b0cca] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18) [0x7b13a8] Mon Jun 8 22:41:47.422 /usr/bin/mongod(main+0x29a) [0x79b89a] Mon Jun 8 22:41:47.422 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f68963cfead] Mon Jun 8 22:41:47.422 /usr/bin/mongod() [0x7adcc9] Mon Jun 8 22:41:47.422 Engine status function not available

Mon Jun 8 22:41:47.422 Memory usage:

Arena 0: system bytes = 0 in use bytes = 0 Total (incl. mmap): system bytes = 0 in use bytes = 0 max mmap regions = 0 max mmap bytes = 0 Backtrace: (Note: toku_do_assert=0x0x7f6897947d90) /usr/lib/tokumx/libtokuportability.so(_Z19db_env_do_backtraceP8_IO_FILE+0x1b)[0x7f68979476bb] /usr/bin/mongod[0x989228] /usr/bin/mongod(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x2b)[0x98c5eb] /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0xa7)[0x7ae1d7] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0)[0x7f68963e31e0] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7f68963e3165] /lib/x86_64-linux-gnu/libc.so.6(abort+0x180)[0x7f68963e63e0] /usr/bin/mongod(_ZN5mongo13fassertFailedEi+0x87)[0xacd247] /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0x1f6)[0x87af86] /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd)[0x87b5bd] /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a)[0x7b0cca] /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18)[0x7b13a8] /usr/bin/mongod(main+0x29a)[0x79b89a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7f68963cfead] /usr/bin/mongod[0x7adcc9] Engine status function not available Memory usage: Arena 0: system bytes = 0 in use bytes = 0 Total (incl. mmap): system bytes = 0 in use bytes = 0 max mmap regions = 0 max mmap bytes = 0 Mon Jun 8 22:41:47.427 Mon Jun 8 22:41:47.427 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.427 Crash reason: Mon Jun 8 22:41:47.427 Mon Jun 8 22:41:47.427 Got signal: 6 (Aborted). Mon Jun 8 22:41:47.427 Mon Jun 8 22:41:47.427 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.427 Process info: Mon Jun 8 22:41:47.427 Mon Jun 8 22:41:47.427 OS: Linux x86_64 Mon Jun 8 22:41:47.427 NCPU: 24 Mon Jun 8 22:41:47.427 VIRT: 824 MB Mon Jun 8 22:41:47.427 RES: 824 MB Mon Jun 8 22:41:47.427 PHYS: 129155 MB Mon Jun 8 22:41:47.427 RLIMIT_CORE: 0 (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_CPU: unlimited (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_DATA: unlimited (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_FSIZE: unlimited (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_NOFILE: 655360 (soft), 655360 (hard) Mon Jun 8 22:41:47.427 RLIMIT_STACK: 8388608 (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_AS: unlimited (soft), unlimited (hard) Mon Jun 8 22:41:47.427 _SC_OPEN_MAX: 655360 Mon Jun 8 22:41:47.427 _SC_PAGESIZE: 4096 Mon Jun 8 22:41:47.427 _SC_PHYS_PAGES: 33063826 Mon Jun 8 22:41:47.427 _SC_AVPHYS_PAGES: 17045999 Mon Jun 8 22:41:47.428 _SC_NPROCESSORS_CONF: 24 Mon Jun 8 22:41:47.428 _SC_NPROCESSORS_ONLN: 24 Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.428 Parsed server options: Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 config: "/data/data/tokumx-cluster/shard2/tokumx.conf" Mon Jun 8 22:41:47.428 dbpath: "/data/data/tokumx-cluster/shard2/data" Mon Jun 8 22:41:47.428 logappend: "true" Mon Jun 8 22:41:47.428 logpath: "/data/data/tokumx-cluster/shard2/log/shard2.log" Mon Jun 8 22:41:47.428 pluginsDir: "/usr/lib/tokumx/plugins" Mon Jun 8 22:41:47.428 port: 27002 Mon Jun 8 22:41:47.428 replSet: "shard2" Mon Jun 8 22:41:47.428 shardsvr: "true" Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.428 Filesystem information: Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 Information for dbpath "/data/data/tokumx-cluster/shard2/data": Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 type magic: 0x0000EF53 Mon Jun 8 22:41:47.428 type: ext2/3/4 Mon Jun 8 22:41:47.428 bsize: 4096 Mon Jun 8 22:41:47.428 blocks: 274983766 Mon Jun 8 22:41:47.428 bfree: 262571347 Mon Jun 8 22:41:47.428 bavail: 248597216 Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.428 Current operations in progress: Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 { opid: 0, active: false, op: "none", ns: "", query: {}, desc: "initandlisten", threadId: "0x7f68983d5800", rootTxnid: 0, waitingForLock: false, lockStats: { timeLockedMicros: {}, timeAcquiringMicros: {} } } Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 -------------------------------------------------------------------------------- Mon Jun 8 22:41:47.428 OpDebug info: Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 none keyUpdates:0 0ms Mon Jun 8 22:41:47.428

esmet commented 9 years ago

Says there may be corruption, so it wont't start.

On Mon, Jun 8, 2015 at 10:50 AM, Huang Diandian notifications@github.com wrote:

when I run, I only got aborted

~$ /usr/bin/mongod --config /data/data/tokumx-cluster/shard2/tokumx.conf Mon Jun 8 22:41:47.280 warning: Mon Jun 8 22:41:47.280 warning: No value set for expireOplogDays, using default of 14 days. Mon Jun 8 22:41:47.280 warning: Mon Jun 8 22:41:47.280 nssize is a deprecated parameter all output going to: /data/data/tokumx-cluster/shard2/log/shard2.log Aborted

log is below:

* SERVER RESTARTED *

Mon Jun 8 22:41:47.285 [initandlisten] TokuMX starting : pid=20477 port=27002 dbpath=/data/data/tokumx-cluster/shard2/data 64-bit host=tokumx01 Mon Jun 8 22:41:47.285 [initandlisten] TokuMX mongod server v2.0.1-mongodb-2.4.10, using TokuKV rev 39fefcc4d2b1421ea09a9e41b130a64b230583b7 Mon Jun 8 22:41:47.285 [initandlisten] git version: d454707 https://github.com/Tokutek/mongo/commit/d4547074dcc9129f2e13ed6b76fd78edb0a2828e Mon Jun 8 22:41:47.285 [initandlisten] build info: Linux 62691929ac33 3.11.0-26-generic #45 https://github.com/Tokutek/mongo/issues/45-Ubuntu SMP Tue Jul 15 04:02:06 UTC 2014 x86_64 GNU/Linux BOOST_LIB_VERSION=1_49 Mon Jun 8 22:41:47.285 [initandlisten] options: { config: "/data/data/tokumx-cluster/shard2/tokumx.conf", dbpath: "/data/data/tokumx-cluster/shard2/data", logappend: "true", logpath: "/data/data/tokumx-cluster/shard2/log/shard2.log", pluginsDir: "/usr/lib/tokumx/plugins", port: 27002, replSet: "shard2", shardsvr: "true" } Mon Jun 8 22:41:47.285 [initandlisten] [tokumx] startup Mon Jun 8 22:41:47 2015 TokuFT recovery starting in env /data/data/tokumx-cluster/shard2/data Mon Jun 8 22:41:47 2015 TokuFT recovery failed -30975 Mon Jun 8 22:41:47.402 [initandlisten] Assertion: 16762:Automatic environment recovery failed. There may be data corruption. 0xc4ced3 0x880f0b 0x882075 0x87a600 0x87ad9f 0x87b5bd 0x7b0cca 0x7b13a8 0x79b89a 0x7f68963cfead 0x7adcc9 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x23) [0xc4ced3] /usr/bin/mongod(_ZN5mongo7storage21MsgAssertionExceptionC2EiRKSs+0x9b) [0x880f0b] /usr/bin/mongod(_ZN5mongo7storage23DataCorruptionExceptionC2EiRKSs+0xa5) [0x882075] /usr/bin/mongod(_ZN5mongo7storage16handle_ydb_errorEi+0x6e0) [0x87a600] /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0xf) [0x87ad9f] /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd) [0x87b5bd] /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a) [0x7b0cca] /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18) [0x7b13a8] /usr/bin/mongod(main+0x29a) [0x79b89a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f68963cfead] /usr/bin/mongod() [0x7adcc9] Mon Jun 8 22:41:47.408 [initandlisten] fatal error 16762: Automatic environment recovery failed. There may be data corruption. Mon Jun 8 22:41:47.408 [initandlisten] 16762 Automatic environment recovery failed. There may be data corruption. Mon Jun 8 22:41:47.408 [initandlisten] Fatal Assertion 16762 0xc4ced3 0xacd20c 0x87af86 0x87b5bd 0x7b0cca 0x7b13a8 0x79b89a 0x7f68963cfead 0x7adcc9 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x23) [0xc4ced3] /usr/bin/mongod(_ZN5mongo13fassertFailedEi+0x4c) [0xacd20c] /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0x1f6) [0x87af86] /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd) [0x87b5bd] /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a) [0x7b0cca] /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18) [0x7b13a8] /usr/bin/mongod(main+0x29a) [0x79b89a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f68963cfead] /usr/bin/mongod() [0x7adcc9] Mon Jun 8 22:41:47.412 [initandlisten]

***aborting after fassert() failure

Mon Jun 8 22:41:47.412

Mon Jun 8 22:41:47.412

Mon Jun 8 22:41:47.412 Fatal error detected

Mon Jun 8 22:41:47.412

Mon Jun 8 22:41:47.412 Mon Jun 8 22:41:47.412 About to gather debugging information, please include all of the following along Mon Jun 8 22:41:47.412 with logs from other servers in the cluster in a bug report. Mon Jun 8 22:41:47.412

Mon Jun 8 22:41:47.412

Mon Jun 8 22:41:47.412 Version info: Mon Jun 8 22:41:47.412 Mon Jun 8 22:41:47.412 tokumxVersion: 2.0.1 Mon Jun 8 22:41:47.412 gitVersion: d454707 https://github.com/Tokutek/mongo/commit/d4547074dcc9129f2e13ed6b76fd78edb0a2828e Mon Jun 8 22:41:47.412 tokukvVersion: 39fefcc4d2b1421ea09a9e41b130a64b230583b7 Mon Jun 8 22:41:47.412 sysInfo: Linux 62691929ac33 3.11.0-26-generic #45 https://github.com/Tokutek/mongo/issues/45-Ubuntu SMP Tue Jul 15 04:02:06 UTC 2014 x86_64 GNU/Linux BOOST_LIB_VERSION=1_49 Mon Jun 8 22:41:47.412 loaderFlags: Mon Jun 8 22:41:47.412 compilerFlags: -fPIC -fno-strict-aliasing -ggdb -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -pipe -Wnon-virtual-dtor -Woverloaded-virtual -Wno-unused-local-typedefs -fno-builtin-memcmp -O3 Mon Jun 8 22:41:47.412 debug: false Mon Jun 8 22:41:47.412

Mon Jun 8 22:41:47.412

Mon Jun 8 22:41:47.412 Simple stacktrace: Mon Jun 8 22:41:47.412 0xc4ced3 0x989157 0x98c5eb 0x7ae1d7 0x7f68963e31e0 0x7f68963e3165 0x7f68963e63e0 0xacd247 0x87af86 0x87b5bd 0x7b0cca 0x7b13a8 0x79b89a 0x7f68963cfead 0x7adcc9 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x23) [0xc4ced3] /usr/bin/mongod() [0x989157] /usr/bin/mongod(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x2b) [0x98c5eb] /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0xa7) [0x7ae1d7] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f68963e31e0] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f68963e3165] /lib/x86_64-linux-gnu/libc.so.6(abort+0x180) [0x7f68963e63e0] /usr/bin/mongod(_ZN5mongo13fassertFailedEi+0x87) [0xacd247] /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0x1f6) [0x87af86] /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd) [0x87b5bd] /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a) [0x7b0cca] /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18) [0x7b13a8] /usr/bin/mongod(main+0x29a) [0x79b89a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f68963cfead] /usr/bin/mongod() [0x7adcc9] Mon Jun 8 22:41:47.417

Mon Jun 8 22:41:47.417

Mon Jun 8 22:41:47.417 TokuKV engine backtrace: Mon Jun 8 22:41:47.417 Mon Jun 8 22:41:47.417 Backtrace: (Note: toku_do_assert=0x0x7f6897947d90)

Mon Jun 8 22:41:47.422 /usr/lib/tokumx/libtokuportability.so( _Z27db_env_do_backtraceerrfuncPFvPKviPKczES0+0x1f) [0x7f6897949a9f] Mon Jun 8 22:41:47.422 /usr/lib/tokumx/libtokufractaltree.so(+0x14bdd) [0x7f6897b63bdd] Mon Jun 8 22:41:47.422 /usr/bin/mongod() [0x989228] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x2b) [0x98c5eb] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0xa7) [0x7ae1d7] Mon Jun 8 22:41:47.422 /lib/x86_64-linux-gnu/libc.so.6(+0x321e0) [0x7f68963e31e0] Mon Jun 8 22:41:47.422 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f68963e3165] Mon Jun 8 22:41:47.422 /lib/x86_64-linux-gnu/libc.so.6(abort+0x180) [0x7f68963e63e0] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo13fassertFailedEi+0x87) [0xacd247] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0x1f6) [0x87af86] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd) [0x87b5bd] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a) [0x7b0cca] Mon Jun 8 22:41:47.422 /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18) [0x7b13a8] Mon Jun 8 22:41:47.422 /usr/bin/mongod(main+0x29a) [0x79b89a] Mon Jun 8 22:41:47.422 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f68963cfead] Mon Jun 8 22:41:47.422 /usr/bin/mongod() [0x7adcc9] Mon Jun 8 22:41:47.422 Engine status function not available

Mon Jun 8 22:41:47.422 Memory usage:

Arena 0: system bytes = 0 in use bytes = 0 Total (incl. mmap): system bytes = 0 in use bytes = 0 max mmap regions = 0 max mmap bytes = 0 Backtrace: (Note: toku_do_assert=0x0x7f6897947d90)

/usr/lib/tokumx/libtokuportability.so(_Z19db_env_do_backtraceP8_IO_FILE+0x1b)[0x7f68979476bb] /usr/bin/mongod[0x989228] /usr/bin/mongod(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x2b)[0x98c5eb] /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0xa7)[0x7ae1d7] /lib/x86_64-linux-gnu/libc.so.6(+0x321e0)[0x7f68963e31e0] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7f68963e3165] /lib/x86_64-linux-gnu/libc.so.6(abort+0x180)[0x7f68963e63e0] /usr/bin/mongod(_ZN5mongo13fassertFailedEi+0x87)[0xacd247]

/usr/bin/mongod(_ZN5mongo7storage22handle_ydb_error_fatalEi+0x1f6)[0x87af86]

/usr/bin/mongod(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x5bd)[0x87b5bd] /usr/bin/mongod(_ZN5mongo14_initAndListenEi+0x34a)[0x7b0cca] /usr/bin/mongod(_ZN5mongo13initAndListenEi+0x18)[0x7b13a8] /usr/bin/mongod(main+0x29a)[0x79b89a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7f68963cfead] /usr/bin/mongod[0x7adcc9] Engine status function not available Memory usage: Arena 0: system bytes = 0 in use bytes = 0 Total (incl. mmap): system bytes = 0 in use bytes = 0 max mmap regions = 0 max mmap bytes = 0 Mon Jun 8 22:41:47.427

Mon Jun 8 22:41:47.427

Mon Jun 8 22:41:47.427 Crash reason: Mon Jun 8 22:41:47.427 Mon Jun 8 22:41:47.427 Got signal: 6 (Aborted). Mon Jun 8 22:41:47.427

Mon Jun 8 22:41:47.427

Mon Jun 8 22:41:47.427 Process info: Mon Jun 8 22:41:47.427 Mon Jun 8 22:41:47.427 OS: Linux x86_64 Mon Jun 8 22:41:47.427 NCPU: 24 Mon Jun 8 22:41:47.427 VIRT: 824 MB Mon Jun 8 22:41:47.427 RES: 824 MB Mon Jun 8 22:41:47.427 PHYS: 129155 MB Mon Jun 8 22:41:47.427 RLIMIT_CORE: 0 (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_CPU: unlimited (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_DATA: unlimited (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_FSIZE: unlimited (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_NOFILE: 655360 (soft), 655360 (hard) Mon Jun 8 22:41:47.427 RLIMIT_STACK: 8388608 (soft), unlimited (hard) Mon Jun 8 22:41:47.427 RLIMIT_AS: unlimited (soft), unlimited (hard) Mon Jun 8 22:41:47.427 _SC_OPEN_MAX: 655360 Mon Jun 8 22:41:47.427 _SC_PAGESIZE: 4096 Mon Jun 8 22:41:47.427 _SC_PHYS_PAGES: 33063826 Mon Jun 8 22:41:47.427 _SC_AVPHYS_PAGES: 17045999 Mon Jun 8 22:41:47.428 _SC_NPROCESSORS_CONF: 24 Mon Jun 8 22:41:47.428 _SC_NPROCESSORS_ONLN: 24 Mon Jun 8 22:41:47.428

Mon Jun 8 22:41:47.428

Mon Jun 8 22:41:47.428 Parsed server options: Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 config: "/data/data/tokumx-cluster/shard2/tokumx.conf" Mon Jun 8 22:41:47.428 dbpath: "/data/data/tokumx-cluster/shard2/data" Mon Jun 8 22:41:47.428 logappend: "true" Mon Jun 8 22:41:47.428 logpath: "/data/data/tokumx-cluster/shard2/log/shard2.log" Mon Jun 8 22:41:47.428 pluginsDir: "/usr/lib/tokumx/plugins" Mon Jun 8 22:41:47.428 port: 27002 Mon Jun 8 22:41:47.428 replSet: "shard2" Mon Jun 8 22:41:47.428 shardsvr: "true" Mon Jun 8 22:41:47.428

Mon Jun 8 22:41:47.428

Mon Jun 8 22:41:47.428 Filesystem information: Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 Information for dbpath "/data/data/tokumx-cluster/shard2/data": Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 type magic: 0x0000EF53 Mon Jun 8 22:41:47.428 type: ext2/3/4 Mon Jun 8 22:41:47.428 bsize: 4096 Mon Jun 8 22:41:47.428 blocks: 274983766 Mon Jun 8 22:41:47.428 bfree: 262571347 Mon Jun 8 22:41:47.428 bavail: 248597216 Mon Jun 8 22:41:47.428

Mon Jun 8 22:41:47.428

Mon Jun 8 22:41:47.428 Current operations in progress: Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 { opid: 0, active: false, op: "none", ns: "", query: {}, desc: "initandlisten", threadId: "0x7f68983d5800", rootTxnid: 0, waitingForLock: false, lockStats: { timeLockedMicros: {}, timeAcquiringMicros: {} } } Mon Jun 8 22:41:47.428

Mon Jun 8 22:41:47.428

Mon Jun 8 22:41:47.428 OpDebug info: Mon Jun 8 22:41:47.428 Mon Jun 8 22:41:47.428 none keyUpdates:0 0ms Mon Jun 8 22:41:47.428

— Reply to this email directly or view it on GitHub https://github.com/Tokutek/mongo/issues/1229.

fydot commented 9 years ago

Are there any way to repair the corruption Or make the server to be start ?