Eternal recursion in ham_cursor_find

[georgehazan]

Hamsterdb 2.1.10 falls into the endless recursion if I declare an environment with the HAM_ENABLE_TRANSACTIONS flag, but the cursor itself has no associated transaction (also there're no active transactions around):

ham_cursor_create(&cursor, _dbi, NULL, 0); ham_cursor_find(cursor, &key, &rec, HAM_FIND_GEQ_MATCH);

Dbx_kv.dll!hamsterdb::LocalDatabase::find_txn(hamsterdb::Context * context=0x000000000012f128, hamsterdb::Cursor * cursor=0x0000000003510060, ham_key_t * key=0x000000000012f468, ham_record_t * record=0x000000000012f428, unsigned int flags=24576) Line 438 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::find_txn(hamsterdb::Context * context=0x000000000012f128, hamsterdb::Cursor * cursor=0x0000000003510060, ham_key_t * key=0x000000000012f468, ham_record_t * record=0x000000000012f428, unsigned int flags=24576) Line 438 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::find_txn(hamsterdb::Context * context=0x000000000012f128, hamsterdb::Cursor * cursor=0x0000000003510060, ham_key_t * key=0x000000000012f468, ham_record_t * record=0x000000000012f428, unsigned int flags=24576) Line 438 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::find_txn(hamsterdb::Context * context=0x000000000012f128, hamsterdb::Cursor * cursor=0x0000000003510060, ham_key_t * key=0x000000000012f468, ham_record_t * record=0x000000000012f428, unsigned int flags=24576) Line 438 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::find_impl(hamsterdb::Context * context=0x000000000012f128, hamsterdb::Cursor * cursor=0x0000000003510060, ham_key_t * key=0x000000000012f468, ham_record_t * record=0x000000000012f428, unsigned int flags=24576) Line 1670 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::find(hamsterdb::Cursor * cursor=0x0000000003510060, hamsterdb::Transaction * txn=0x0000000000000000, ham_key_t * key=0x000000000012f468, ham_record_t * record=0x000000000012f428, unsigned int flags=24576) Line 1135 C++ Dbx_kv.dll!ham_cursor_find(ham_cursor_t * hcursor=0x0000000003510060, ham_key_t * key=0x000000000012f468, ham_record_t * record=0x000000000012f428, unsigned int flags=24576) Line 1278 C++ Dbx_kv.dll!CDbxKV::EnumContactSettings(unsigned int contactID=0, DBCONTACTENUMSETTINGS * dbces=0x000000000012f6f8) Line 585 C++

[georgehazan]

PS: there's no such a key in a database

[cruppstahl]

Thanks for reporting. I reviewed the code and managed to create two test cases which are problematic, but only when doing a LEQ search and they do not cause endless loops.

Can you share some code which reproduces the problem?

2015-04-01 18:59 GMT+02:00 George Hazan notifications@github.com:

PS: there's no such a key in a database

— Reply to this email directly or view it on GitHub https://github.com/cruppstahl/hamsterdb/issues/50#issuecomment-88542876.

[georgehazan]

thanks for the quick answer. it's a part of quite a complex database, I'll try to make a simple snippet that illustrates that problem

[georgehazan]

sigh I have to confess that this problem is not reproducible at all in the static environment. it occurs only when the massive array of transactions is being fed to hamsterdb in the same time with reading. I can reproduce this problem easily, but it requires a working program. feel free to ask me if I could provide the interesting information.

[cruppstahl]

I am not surprised... Approx. matching is covered with many tests, but it has some complexities. I will try to come up with a fix based on the callstack, maybe you could then help testing it. Are you building from source, or do you require prebuilt windows libraries? DLL or static, msvc2010 or -13, 32bit or 64bit?

Thanks Christoph

2015-04-02 13:55 GMT+02:00 George Hazan notifications@github.com:

sigh I have to confess that this problem is not reproducible at all in the static environment. it occurs only when the massive array of transactions is being fed to hamsterdb in the same time with reading. I can reproduce this problem easily, but it requires a working program. feel free to ask me if I could provide the interesting information.

— Reply to this email directly or view it on GitHub https://github.com/cruppstahl/hamsterdb/issues/50#issuecomment-88877067.

[georgehazan]

I'm building hamsterdb from sources, yes, embedding it into the Miranda database driver

On Thu, Apr 2, 2015 at 3:53 PM, Christoph Rupp notifications@github.com wrote:

I am not surprised... Approx. matching is covered with many tests, but it has some complexities. I will try to come up with a fix based on the callstack, maybe you could then help testing it. Are you building from source, or do you require prebuilt windows libraries? DLL or static, msvc2010 or -13, 32bit or 64bit?

Thanks Christoph

2015-04-02 13:55 GMT+02:00 George Hazan notifications@github.com:

sigh I have to confess that this problem is not reproducible at all in the static environment. it occurs only when the massive array of transactions is being fed to hamsterdb in the same time with reading. I can reproduce this problem easily, but it requires a working program. feel free to ask me if I could provide the interesting information.

— Reply to this email directly or view it on GitHub <https://github.com/cruppstahl/hamsterdb/issues/50#issuecomment-88877067 .

— Reply to this email directly or view it on GitHub https://github.com/cruppstahl/hamsterdb/issues/50#issuecomment-88887000.

WMBR, George Hazan

[cruppstahl]

Can you please try the newest commit? It's on the "v2" branch, not on "master".

[georgehazan]

sure

[georgehazan]

msvcr120d.dll!_wassert()   Unknown

Dbx_kv.dll!hamsterdb::rbt_next(hamsterdb::TransactionIndex * rbtree=0x000000000029df70, hamsterdb::TransactionNode * node=0x000000000486e940) Line 62 C++ Dbx_kv.dll!hamsterdb::TransactionIndex::remove(hamsterdb::TransactionNode * node=0x00000000048471b0) Line 218 C++ Dbx_kv.dll!hamsterdb::TransactionOperation::destroy() Line 108 C++ Dbx_kv.dll!hamsterdb::TransactionFactory::destroy_operation(hamsterdb::TransactionOperation * op=0x0000000004847250) Line 58 C++ Dbx_kv.dll!hamsterdb::LocalTransaction::free_operations() Line 302 C++ Dbx_kv.dll!hamsterdb::LocalTransaction::~LocalTransaction() Line 260 C++ Dbx_kv.dll!hamsterdb::LocalTransaction::`scalar deleting destructor'(unsigned int) C++ Dbx_kv.dll!hamsterdb::LocalTransactionManager::flush_committed_txns_impl(hamsterdb::Context * context=0x000000000012ef48) Line 621 C++ Dbx_kv.dll!hamsterdb::LocalTransactionManager::maybe_flush_committed_txns(hamsterdb::Context * context=0x000000000012ef48) Line 567 C++ Dbx_kv.dll!hamsterdb::LocalTransactionManager::commit(hamsterdb::Transaction * htxn=0x000000000486e7b0, unsigned int flags=0) Line 521 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::finalize(hamsterdb::Context * context=0x000000000012f088, int status=0, hamsterdb::Transaction * local_txn=0x000000000486e7b0) Line 1831 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::insert(hamsterdb::Cursor * cursor=0x0000000000000000, hamsterdb::Transaction * txn=0x0000000000000000, ham_key_t * key=0x000000000012f478, ham_record_t * record=0x000000000012f4a8, unsigned int flags=1) Line 1124 C++ Dbx_kv.dll!ham_db_insert(ham_db_t * hdb=0x000000000029dc80, ham_txn_t * htxn=0x0000000000000000, ham_key_t * key=0x000000000012f478, ham_record_t * record=0x000000000012f4a8, unsigned int flags=1) Line 984 C++ Dbx_kv.dll!CDbxKV::WriteContactSetting(unsigned int contactID=0, DBCONTACTWRITESETTING * dbcws=0x000000000012f5d8) Line 511 C++ mir_core.dll!db_set_dw(unsigned int hContact=0, const char * szModule=0x00000000055508e8, const char * szSetting=0x00000000055508d8, unsigned long val=10) Line 204 C++

[georgehazan]

stopped on ASSERT

[cruppstahl]

Can you tell me which assert exactly? the one in db_local.cc:478?

2015-04-06 13:45 GMT+02:00 George Hazan notifications@github.com:

stopped on ASSERT

— Reply to this email directly or view it on GitHub https://github.com/cruppstahl/hamsterdb/issues/50#issuecomment-90026769.

[georgehazan]

[cruppstahl]

If you could also send a callstack, that would be extremely helpful. Sorry for the efforts.

[georgehazan]

yes, the call stack is pasted above, 43 minutes ago

[cruppstahl]

I have pushed another commit to the v2 branch. It removes the code which triggered the assert. I think the assert was problematic. However, if this still does not work then i'll start refactoring larger parts of the code base and clean up some things which led to this bug.

[georgehazan]

okay

[georgehazan]

Dbx_kv.dll!hamsterdb::rbt_remove(hamsterdb::TransactionIndex * rbtree=0x00000000002baa90, hamsterdb::TransactionNode * node=0x00000000047a36f0) Line 62 C++ Dbx_kv.dll!hamsterdb::TransactionIndex::remove(hamsterdb::TransactionNode * node=0x00000000047a36f0) Line 211 C++ Dbx_kv.dll!hamsterdb::TransactionOperation::destroy() Line 108 C++ Dbx_kv.dll!hamsterdb::TransactionFactory::destroy_operation(hamsterdb::TransactionOperation * op=0x00000000047ba1f0) Line 58 C++ Dbx_kv.dll!hamsterdb::LocalTransaction::free_operations() Line 289 C++ Dbx_kv.dll!hamsterdb::LocalTransaction::~LocalTransaction() Line 247 C++ Dbx_kv.dll!hamsterdb::LocalTransaction::`scalar deleting destructor'(unsigned int) C++ Dbx_kv.dll!hamsterdb::LocalTransactionManager::flush_committed_txns_impl(hamsterdb::Context * context=0x000000000012ef48) Line 608 C++ Dbx_kv.dll!hamsterdb::LocalTransactionManager::maybe_flush_committed_txns(hamsterdb::Context * context=0x000000000012ef48) Line 554 C++ Dbx_kv.dll!hamsterdb::LocalTransactionManager::commit(hamsterdb::Transaction * htxn=0x00000000047bd900, unsigned int flags=0) Line 508 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::finalize(hamsterdb::Context * context=0x000000000012f088, int status=0, hamsterdb::Transaction * local_txn=0x00000000047bd900) Line 1834 C++ Dbx_kv.dll!hamsterdb::LocalDatabase::insert(hamsterdb::Cursor * cursor=0x0000000000000000, hamsterdb::Transaction * txn=0x0000000000000000, ham_key_t * key=0x000000000012f478, ham_record_t * record=0x000000000012f4a8, unsigned int flags=1) Line 1127 C++ Dbx_kv.dll!ham_db_insert(ham_db_t * hdb=0x00000000002ba7a0, ham_txn_t * htxn=0x0000000000000000, ham_key_t * key=0x000000000012f478, ham_record_t * record=0x000000000012f4a8, unsigned int flags=1) Line 984 C++ Dbx_kv.dll!CDbxKV::WriteContactSetting(unsigned int contactID=0, DBCONTACTWRITESETTING * dbcws=0x000000000012f5d8) Line 511 C++ mir_core.dll!db_set_dw(unsigned int hContact=0, const char * szModule=0x00000000055408e8, const char * szSetting=0x00000000055408d8, unsigned long val=10) Line 204 C++

[cruppstahl]

OK thanks. I'll get back at you when a newer version is in a sufficiently good state - 1 or 2 weeks at least.

[georgehazan]

sure. feel free for pinging me