search

kriszyp / lmdb-js

Simple, efficient, ultra-fast, scalable data store wrapper for LMDB
Other
481 stars 39 forks source link

Deadlock (?) after switching to 2.7.0 #203

Closed ppedziwiatr closed 1 year ago

ppedziwiatr commented 1 year ago

Hey,

my application seems to hang completely after switching to lmdb-js version 2.7.0.

Version 2.6.9 works properly. No other changes have been made.

Do you have any idea what might be causing the issues?

kriszyp commented 1 year ago

There were some changes to how locks are used to wait for transactions to be flushed to disk (seems likely related to this), but I am not sure how it would actually end up in a deadlock. Do you have any suggestions for how I could reproduce this?

ppedziwiatr commented 1 year ago

Hey, I will try to prepare some MRE, just need some time to sort out my own stuff ;-)

kriszyp commented 1 year ago

Ok, thanks. And I will do some of our own load testing, and see if I spot anything.

atomictag commented 1 year ago

I noticed the same problem. If that helps, my setup is lmdb-darwin-x64. Version 2.7.x hangs with a minimal test:

const rootDb = lmdb.open(...) // storage is created on file ok
await rootDb.put("something", "anything");

After I reverted to 2.6.9 I noticed that running the same test on a db created with 2.7.0 would leave the promise unresolved. Running the same test with 2.6.9 after deleting the db store would work again as usual.

kriszyp commented 1 year ago

I haven't been able to reproduce this. I did find potential issue with how locks were created on MacOS, and fixed in v2.7.2, if you want to give that a try and see if it happens to make any difference.

atomictag commented 1 year ago

I just tried 2.7.2 and all my tests pass again without any issues 🎉. Thank you!

ppedziwiatr commented 1 year ago

I'm trying to test the 2.7.2 on my M1, unfortunately I'm having this issue while doing yarn install:

error /Users/piotrpedziwiatr/projects/warp-contracts-lmdb/node_modules/lmdb: Command failed.
Exit code: 127
Command: node-gyp-build-optional-packages
Arguments:
Directory: /Users/piotrpedziwiatr/projects/warp-contracts-lmdb/node_modules/lmdb
Output:
env: node\r: No such file or directory

Tested on Node 18.5 and 16.16.

Version 2.7.1 installs normally...

ppedziwiatr commented 1 year ago

hmm, npm install worked properly, so it's probably sth with yarn...

kriszyp commented 1 year ago

Ugh, those line ending errors are the worst. Should be fixed with 2.7.3, I hope.

ppedziwiatr commented 1 year ago

Yup, it's working great, thank you very much :-) deadlock is also fixed on my M1