search

kriszyp / lmdb-js

Simple, efficient, ultra-fast, scalable data store wrapper for LMDB
Other
484 stars 39 forks source link

Module does not self-register lmdb-store.node #67

Closed rizrmd closed 2 years ago

rizrmd commented 3 years ago

I am using lmdb-store with chokidar and restart node proces when my source file changes. At first lmdb-store runs fine, and then I got this error when live-reload occured: 

Error: Module did not self-register: '/Users/riz/Documents/base/node_modules/lmdb/build/Release/lmdb-store.node'.
 at Object.Module._extensions..node (node:internal/modules/cjs/loader:1154:18)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:816:12)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (node:internal/modules/cjs/helpers:93:18)
    at load (/Users/riz/Documents/base/node_modules/node-gyp-build/index.js:21:10)
    at Object.<anonymous> (/Users/riz/Documents/base/node_modules/lmdb/index.js:7:49)
    at Module._compile (node:internal/modules/cjs/loader:1095:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1124:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
Emitted 'error' event on Thread instance at:
    at Thread.[kOnErrorMessage] (node:internal/worker:298:10)
    at Thread.[kOnMessage] (node:internal/worker:309:37)
    at MessagePort.<anonymous> (node:internal/worker:210:57)
    at MessagePort.[nodejs.internal.kHybridDispatch] (node:internal/event_target:461:20)
    at MessagePort.exports.emitMessage (node:internal/per_context/messageport:23:28) {
  code: 'ERR_DLOPEN_FAILED'
}

I am using macbook m1 fwiw, is there anything I can do to solve this ?

kriszyp commented 3 years ago

Do you know how you are restarting the node process? Is it completely killing the process and restarting it? I think chokidar is just responsible for file watching, not restarting.

I did update the code to use the latest recommended node add-on API and published as part of v1.6.0-beta.1 if you want to try that and see if it makes a difference?

rizrmd commented 3 years ago

I am using worker_thread.terminate() method to kill the process. Ok, will try tomorrow maybe. I just switched using sqlite for meantime...

rizrmd commented 3 years ago

So the way I run lmdb-store is like this:

Main process will restart db worker thread on file change

kriszyp commented 3 years ago

I have not been able to reproduce this while terminating and starting new threads. Does it make any difference if the main process has loaded/required lmdb-store (I think there can be a different cleanup procedure if the module is still in use)? Also what version of node is this?

rizrmd commented 3 years ago

This is latest node v.16.4.0, Yeah now it's intermitently terminated. On a bad day, almost every reload is terminated.

rizrmd commented 3 years ago

FWIW, I use esbuild to build my ts source file, and then run resulting build file using worker_threads. So I execute different source code for each run (esbuild generate new file whenever file changes)

rizrmd commented 3 years ago

I have not been able to reproduce this while terminating and starting new threads. Does it make any difference if the main process has loaded/required lmdb-store (I think there can be a different cleanup procedure if the module is still in use)? Also what version of node is this?

When I use node-lmdb on main, it's running smooth. But, I really need to run node-lmdb on worker_thread because we have several thread to process different database, and can kill it whenever we need without interupting main process. Also we use worker_thread because we can just postMessage when we thread needs to coordinate each other.

kriszyp commented 3 years ago

When I use node-lmdb on main, it's running smooth. But, I really need to run node-lmdb on worker_thread

What I meant was if you could just load lmdb-store in the main thread and still use lmdb-store in the worker threads as you are currently doing. It sounds like you have probably already tried that though.

I have tried reproducing this on multiple machines now, including on Windows, Mac, and Linux without any issues or success in reproducing this. I suppose it is possible it is related to M1 (I don't have access to an M1 Mac right now). I have previously observed an issue (#48 ) with M1's apparently having different timing with threads resuming after a mutex unlocks, and node does use mutexes for add-on registration (and threads and non-determinism also are at play). However, that seems like it would affect (all?) other native add-ons running on Node + M1, which seems unlikely to me (surely someone else would have noticed). Is it possible for you to test if this does affect any other native add-ons? lmdb-store should transitively depend-on/install msgpackr-extract, maybe you see if this error occurs if your worker threads load/require the msgpackr-extract package (or other native add-on packages)?

Or if you have any test case that I can run, that would certainly be welcome also.

Anyway, your setup is very legitimate way of running lmdb-store (we basically do the same exact thing in our medical software, except we use processes instead of threads), so I definitely would like to get it fixed.

rizrmd commented 3 years ago

When I use node-lmdb on main, it's running smooth. But, I really need to run node-lmdb on worker_thread

What I meant was if you could just load lmdb-store in the main thread and still use lmdb-store in the worker threads as you are currently doing. It sounds like you have probably already tried that though.

I have tried reproducing this on multiple machines now, including on Windows, Mac, and Linux without any issues or success in reproducing this. I suppose it is possible it is related to M1 (I don't have access to an M1 Mac right now). I have previously observed an issue (#48 ) with M1's apparently having different timing with threads resuming after a mutex unlocks, and node does use mutexes for add-on registration (and threads and non-determinism also are at play). However, that seems like it would affect (all?) other native add-ons running on Node + M1, which seems unlikely to me (surely someone else would have noticed). Is it possible for you to test if this does affect any other native add-ons? lmdb-store should transitively depend-on/install msgpackr-extract, maybe you see if this error occurs if your worker threads load/require the msgpackr-extract package (or other native add-on packages)?

Or if you have any test case that I can run, that would certainly be welcome also.

Anyway, your setup is very legitimate way of running lmdb-store (we basically do the same exact thing in our medical software, except we use processes instead of threads), so I definitely would like to get it fixed.

Well. yeah, M1 support is definitely flaky on npm, so my choices is limited and the best I can do is report like this. It would take me a while to set up a test case. My hand is currently full, my team already past the deadline -- sorry!

I will report back to you when I can touch this issue again.

hgeist2 commented 2 years ago

I have this problem, too, when using parcel 2.0.0+

OS: 5.4.0-89-generic #100-Ubuntu SMP, Ubuntu 20.04 Node: 14.15.4 Npm: 6.14.11

It was not resolved by:

This issue could be related: parcel-bundler/parcel#6597 -> Other issue with other exception

kriszyp commented 2 years ago

@hgeist2 Are you getting a "symbol not found" error or a "Module does not self-register" error?

hgeist2 commented 2 years ago

@kriszyp: Its the "Module does not self-regitser" one

Error: Module did not self-register: '/home/.../node_modules/lmdb-store/build/Release/lmdb-store.node'.

kriszyp commented 2 years ago

Do you have any ideas if there is any other parcel configurations that are likely to help reproduce this? I have a ubuntu box here I can install parcel on and try, but I'm guessing that parcel folks at least tested the basic parcel+ubuntu combination before releasing. Anyway, I would love to be able to track down this mysterious bug though.

hgeist2 commented 2 years ago

I will share with you as much as i can.

I read somewhere that parcel-cache uses lmdb-store.

The repo also uses git submodules.

Full Stack trace:

Error: Module did not self-register: '.../node_modules/lmdb-store/build/Release/lmdb-store.node'. at Object.Module._extensions..node (internal/modules/cjs/loader.js:1122:18) at Module.load (internal/modules/cjs/loader.js:928:32) at Function.Module._load (internal/modules/cjs/loader.js:769:14) at Module.require (internal/modules/cjs/loader.js:952:19) at require (.../node_modules/v8-compile-cache/v8-compile-cache.js:159:20) at load (.../node_modules/node-gyp-build/index.js:21:10) at Object. (.../node_modules/lmdb-store/index.js:7:49) at Module._compile (.../node_modules/v8-compile-cache/v8-compile-cache.js:192:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10) at Module.load (internal/modules/cjs/loader.js:928:32) npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! xyz@1.0.0 start: npm run clean && parcel src/index.pug --no-autoinstall npm ERR! Exit status 1

package.json

{
  "name": "xyz",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "debug_build": "NODE_ENV=development parcel build src/index.pug --no-optimize",
    "build": "parcel build src/index.pug --no-source-maps",
    "clean": "rimraf .parcel-cache dist",
    "deploy": "deploy-aws-s3-cloudfront --non-interactive --delete --source ./dist --bucket $npm_package_config_bucket_name --distribution $npm_package_config_distribution_id --destination /$npm_package_config_public_url --invalidation-path /$npm_package_config_public_url/* --cache-control index.html:no-cache",
    "format": "prettier \"src/**/*.+(js|jsx|json|scss|md)\"",
    "format:write": "npm run format -- --write",
    "lint": "npm run lint_js && npm run lint_scss",
    "lint:fix": "npm run lint_js:fix && npm run lint_scss:fix",
    "lint_js": "eslint --ext .js,.jsx,.ts,.tsx src",
    "lint_js:fix": "npm run lint_js -- --fix",
    "lint_scss": "stylelint 'src/**/*.scss'",
    "lint_scss:fix": "npm run lint_scss -- --fix",
    "predeploy": "npm run build",
    "start": "npm run clean && parcel src/index.pug --no-autoinstall",
    "start_debug": "serve dist/",
    "test": "jest --roots src",
    "test_u": "npm run test -- -u",
    "prepare": "husky install"
  },
  "config": {
    "bucket_name": "XYZ",
    "distribution_id": "XYZ",
    "public_url": "example.com"
  },
  "jest": {
    "setupFilesAfterEnv": [
      "./contrib/jest.setup.js"
    ],
    "transform": {
      "^.+\\.jsx?$": "babel-jest"
    },
    "moduleNameMapper": {
      "~/(.*)": "<rootDir>/src/$1",
      "(url)\\:": "jest-transform-stub",
      "\\.(s?css)$": "identity-obj-proxy",
      "(contrib)/?(.*)$": "<rootDir>/$1/$2"
    }
  },
  "alias": {
    "contrib": "./contrib"
  },
  "author": "",
  "license": "",
  "dependencies": {
    "@aws-amplify/auth": "^4.3.9",
    "@google/model-viewer": "1.6.0",
    "@jest/globals": "^26.6.2",
    "@redux-saga/core": "^1.1.3",
    "@testing-library/jest-dom": "^5.12.0",
    "@testing-library/react": "^11.2.7",
    "@tweenjs/tween.js": "^18.6.4",
    "autoprefixer": "^10.2.5",
    "axios": "0.21.1",
    "browserify": "^17.0.0",
    "classnames": "2.3.1",
    "deploy-aws-s3-cloudfront": "3.3.0",
    "hammerjs": "^2.0.8",
    "html2canvas": "^1.1.2",
    "is-ios": "^2.0.1",
    "jest-snapshot": "^26.6.2",
    "lmdb-store": "^1.6.3",
    "material-icons": "^0.6.1",
    "parcel-plugin-asset-copier": "^1.1.0",
    "parse-color": "^1.0.0",
    "react": "17.0.2",
    "react-device-detect": "^2.0.0",
    "react-dom": "17.0.2",
    "react-draggable": "^4.4.3",
    "react-iframe": "1.8.0",
    "react-indexed-db": "^1.0.12",
    "react-nl2br": "1.0.2",
    "react-phone-number-input": "^3.1.21",
    "react-redux": "7.2.3",
    "react-router-dom": "^5.2.0",
    "react-svg": "13.0.2",
    "react-swipeable-views": "0.13.9",
    "redux": "4.1.0",
    "redux-persist": "^6.0.0",
    "redux-saga": "1.1.3",
    "redux-saga-thunk": "0.7.3",
    "serve": "^12.0.1",
    "three": "^0.133.1",
    "uuid": "8.3.2",
    "video-react": "0.14.1"
  },
  "devDependencies": {
    "@babel/eslint-parser": "^7.15.8",
    "@babel/plugin-proposal-decorators": "^7.15.8",
    "@babel/plugin-transform-runtime": "7.13.15",
    "@babel/preset-env": "^7.15.8",
    "@babel/preset-react": "^7.14.5",
    "@parcel/babel-preset-env": "^2.0.0",
    "@parcel/transformer-image": "^2.0.0",
    "@parcel/transformer-pug": "^2.0.0",
    "@parcel/transformer-sass": "^2.0.0",
    "@wojtekmaj/enzyme-adapter-react-17": "0.6.1",
    "babel-jest": "26.6.3",
    "babel-plugin-transform-remove-console": "^6.9.4",
    "enzyme": "3.11.0",
    "eslint": "7.24.0",
    "eslint-config-prettier": "8.2.0",
    "eslint-config-semistandard": "15.0.1",
    "eslint-config-standard": "16.0.2",
    "eslint-import-resolver-typescript": "2.4.0",
    "eslint-plugin-import": "2.22.1",
    "eslint-plugin-node": "11.1.0",
    "eslint-plugin-prettier": "3.3.1",
    "eslint-plugin-promise": "5.1.0",
    "eslint-plugin-react": "7.23.2",
    "eslint-plugin-react-hooks": "4.2.0",
    "history": "4.6.3",
    "husky": "6.0.0",
    "identity-obj-proxy": "3.0.0",
    "jest": "26.6.3",
    "jest-transform-stub": "2.0.0",
    "parcel": "^2.0.0-nightly.893",
    "parcel-plugin-static-files-copy": "^2.6.0",
    "prettier": "2.2.1",
    "prop-types": "^15.7.2",
    "rimraf": "3.0.2",
    "stylelint": "13.12.0",
    "stylelint-config-sass-guidelines": "8.0.0",
    "stylelint-scss": "3.19.0"
  }
}

babel.config.json

{
  "env": {
    "test": {
      "presets": ["@parcel/babel-preset-env", "@babel/preset-react"]
    },
    "development": {
      "presets": ["@parcel/babel-preset-env", "@babel/preset-react"],
      "plugins": [
        "@babel/plugin-transform-runtime"
      ]
    },
    "production": {
      "presets": ["@parcel/babel-preset-env"],
      "plugins": [
        "transform-remove-console"
      ]
    },
  }
}
kriszyp commented 2 years ago

Thank you for the details! I'll see if I can reproduce.

kriszyp commented 2 years ago

@hgeist2 I was able to start this right up with your config files and just adding a index.pug, and it all ran perfectly fine for me on my linux mint computer. Based on previous comments, it sounded like this had something to do with workers/threads, is it possible that there is something that triggers multiple threads to start in parcel in your setup? And this a consistent error, or intermittent?

mischnic commented 2 years ago

By default, Parcel uses worker threads. You can override the automatic calculation with PARCEL_WORKERS=2 parcel build .... With 0, everything runs on the main thread - so if you still get the error in that case, it shouldn't be related to workers.

hgeist2 commented 2 years ago

Turns out updating to node 15.x/15.4 solves the problem.

kriszyp commented 2 years ago

Hmm, that's interesting. Node 15.x isn't really supported (by NodeJS, only 12.x, 14.x, 16.x, and 17.x are the maintained/supported versions). And the OP was using Node 16.x. I wonder if this goes away on a newer 16.x as well?

And my attempts to reproduce this were with Node 14.17.3. And I did actually try PARCEL_WORKERS=20 parcel... but that didn't cause an error either.

hgeist2 commented 2 years ago

If i switch to 16.13.0 it stops working and if i switch back to 15.4.0 it starts working again.

Im not sure PARCEL_WORKERS=x really does anything in parcel 2.0.0, i could not find anything in the docs about it.

hgeist2 commented 2 years ago

The project leader downloaded and tried to compile lmdb-store locally. That went fine, but the tests failed.

Maybe both stacktraces stem from a problem with type casting? Could be confirmed if the test below starts working on 15.4.0

Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:24 PDT 2021; root:xnu-8019.41.5~1/RELEASE_ARM64_T8101 arm64

% npm test
> lmdb-store@1.6.10 test ExternalSource/lmdb-store
> mocha test/**.test.js --recursive && npm run test:types
(node:43728) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
  lmdb-store
    Basic use
FATAL ERROR: v8::Function::Cast Value is not a Function
 1: 0x1022300a4 node::Abort() [.nvm/versions/node/v14.17.4/bin/node]
 2: 0x102230224 node::errors::TryCatchScope::~TryCatchScope() [.nvm/versions/node/v14.17.4/bin/node]
 3: 0x10235aab8 v8::Function::CheckCast(v8::Value*) [.nvm/versions/node/v14.17.4/bin/node]
 4: 0x10fb988c8 v8::Function::Cast(v8::Value*) [ExternalSource/lmdb-store/build/Debug/lmdb-store.node]
 5: 0x10fb8f984 v8::Local<v8::Function> v8::Local<v8::Function>::Cast<v8::Value>(v8::Local<v8::Value>) [ExternalSource/lmdb-store/build/Debug/lmdb-store.node]
 6: 0x10fb8eca0 EnvWrap::open(Nan::FunctionCallbackInfo<v8::Value> const&) [ExternalSource/lmdb-store/build/Debug/lmdb-store.node]
 7: 0x10fb973f4 Nan::imp::FunctionCallbackWrapper(v8::FunctionCallbackInfo<v8::Value> const&) [ExternalSource/lmdb-store/build/Debug/lmdb-store.node]
 8: 0x1023b2108 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [.nvm/versions/node/v14.17.4/bin/node]
 9: 0x1023b1714 v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [.nvm/versions/node/v14.17.4/bin/node]
10: 0x1023b0fac v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [.nvm/versions/node/v14.17.4/bin/node]
11: 0x102acc3ec Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [.nvm/versions/node/v14.17.4/bin/node]
12: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
13: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
14: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
15: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
16: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
17: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
18: 0x102a64ff4 Builtins_InterpreterEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
19: 0x102a628a4 Builtins_JSEntryTrampoline [.nvm/versions/node/v14.17.4/bin/node]
20: 0x102a62548 Builtins_JSEntry [.nvm/versions/node/v14.17.4/bin/node]
21: 0x148008000 
sh: line 1: 43728 Abort trap: 6           mocha test/**.test.js --recursive
npm ERR! Test failed.  See above for more details.
hgeist2 commented 2 years ago

Sorry for the spam.

I downloaded lmdb-store 1.6.11. and ran npm test

That is directly inverted to the self-register stacktrace and node relationship.

kriszyp commented 2 years ago

Sorry for the spam.

I appreciate it, would love to track this down!

So I believe the type casting stacktrace is specific to running the debug builds (in combination with some specific node version and OS, I assume, since I haven't seen it before), which are only run by default with npm run recompile, and not with the prebuilds or default fallback build. I can easily fix this, and will, but doesn't seem likely to be the cause of the self-register (as you noted with the inverted relationship).

I tried your app/config specifically with node v14.15.4 and v16.13.0, and still didn't see any self-register error. I noticed in the other stack trace that it is ARM64; are you on ARM64 too? That might be an interesting bit of commonality with the OP.

hgeist2 commented 2 years ago

I am on x64 / Ryzen. The other stacktrace with ARM64 is a new M1 Mac

hgeist2 commented 2 years ago

Self-register stacktrace prevails on 5.4.0-89-generic #100-Ubuntu SMP, Ubuntu 20.04, x64 Ryzen as above with the following node versions:

Only 15.4.0 works and i can reproduce it. Installations via nvm.

https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V15.md#15.4.0

kriszyp commented 2 years ago

Are you recompiling with each of these nvm changes?

I actually can reproduce this self-register error message by deleting/renaming my prebuilds, compiling with one node version, and then switching to a different node version (with nvm as well). This is because the compiled code is for the wrong node version. This doesn't happen if your app is using the prebuilds, since switching node versions just ends up selecting a different/appropriate prebuild.

I am guessing what happened is that lmdb-store got compiled/built on a node version that doesn't have a prebuild, specifically a v15.x version (I don't include prebuilds for the v15.x line because they aren't supported/maintained by NodeJS). And once lmdb-store gets compiled/built, it uses the locally compiled version over the prebuilds, and so even when you switch node versions, the locally compiled version gets used instead of the appropriate prebuild.

What I don't understand is why v15.x versions would be incompatible with each other, since all 15.x versions are supposed to be on the same ABI.

So I guess the questions is if this error still occurs if lmdb-store is re-installed/compiled with the node version that is currently being used? Or if it occurs with one of the prebuilds (using node 14.x or 16.x with the build/Release directory deleted/renamed)?

hgeist2 commented 2 years ago

That doesnt seem to be it. Shuffling the node js versions between 14.x, 15.x and 16.x and always deleting node_modules/ + npm i after each version switch still leads to same results.

kriszyp commented 2 years ago

Did you happen to notice if it is compiling/building for all the different node js versions (versus using the prebuild, which shouldn't involve any compiling or any files created in build/Release)? I am just curious if there are possibly any differences between locally compiled and prebuilds in regards to this. And does your coworker get the self-register error? Do you have any idea if this is specific to certain OS?

hgeist2 commented 2 years ago

My coworker said they didnt have problem with other parcel 2.0.0 projects and any kind of machine, be it mac m1, windows 10 or mint linux. It seems to be specific to my system.

Some recompiling was done compiling cpp files for lmdb, but i did not watch closely. Will retry again tonight / next days

hgeist2 commented 2 years ago

Maybe this could be relevant to you? https://stackoverflow.com/a/41283828

I searched lmdb-store repo for NODE_SET_METHOD and NODE_MODULE without results.

Maybe it solves automagically if you provide this initialization method additionally to NODE_MODULE_INITIALIZER in node-lmdb.cpp in case some packer like parcel does something weird or use NODE_MODULE_INIT instead of NODE_MODULE_INITIALIZER like they show in their example.

https://nodejs.org/api/addons.html#addons_context_aware_addons

If you provide me with the code i could try a build like that on my machine.

kriszyp commented 2 years ago

Maybe it solves automagically if you provide this initialization method additionally to NODE_MODULE_INITIALIZER in node-lmdb.cpp in case some packer like parcel does something weird or use NODE_MODULE_INIT instead

You can't use NODE_MODULE and NODE_MODULE_INIT, since they both are defining the same function (_module), so they conflict. And using NODE_MODULE instead of the context-aware initializer doesn't work with worker threads (it will throw the self-register error the first time a worker thread tries to load it).

However, in the latest commit, I did try a few things: I switched from NODE_MODULE_INITIALIZER to NODE_MODULE_INIT. I had tried this before and it didn't work, causing compilation errors. I realized this is because the build target name had a dash in it, and the macro was directly copying the target name into the source. I fixed this by change the build target name (to just "lmdb"). With this, I was able to switch to the more standard NODE_MODULE_INIT.

I added some crude logging that can be enabled with LMDB_STORE_LOGGING=true env variable. This should log the steps of starting the module and initializing/registering.

I also saw a note in the SO post about having the registration module be the first in the list of sources. I have never heard that before, but figured I would try that as well.

I did also include a commented-out sample code that use NODE_MODULE in node-ldmb.cpp, if you want to test that (uncomment it, and comment out NODE_MODULE_INIT), but again, that isn't really a viable solution.

This update is in master. Would you be willing to try that? Have you previously been able to reproduce the error just by running the lmdb-store tests? Or do you want me to publish this in NPM?

hgeist2 commented 2 years ago

Thank you for the fast solution, i will try to build it from master and then give you feedback.

vozeldr commented 2 years ago

I have this same issue with node v17.0.1 and npm 8.1.0 running on Pop!_OS 21.04.

My project only has these dependencies and the error is originating from v8-compile-cache

  "dependencies": {
    "tslib": "^2.3.1"
  },
  "devDependencies": {
    "@parcel/config-default": "^2.0.0",
    "@parcel/transformer-typescript-tsc": "^2.0.0",
    "@tsconfig/recommended": "^1.0.1",
    "@types/jest": "^27.0.2",
    "@types/webextension-polyfill": "^0.8.0",
    "copyfiles": "^2.4.1",
    "cross-zip-cli": "^1.0.0",
    "jest": "^27.2.4",
    "jest-webextension-mock": "^3.7.19",
    "npm-run-all": "^4.1.5",
    "parcel": "^2.0.0",
    "rename-files": "^0.0.2",
    "rimraf": "^3.0.2",
    "ts-jest": "^27.0.5",
    "typescript": "^4.4.3",
    "webextension-polyfill": "^0.8.0"
  }
kriszyp commented 2 years ago

@vozeldr Do you happen to know if lmdb-store compiled a build (created a node_modules/lmdb-store/build/Release, otherwise it is using the prebuilds)? Would you be able to try building lmdb-store from master, in case the changes I made for @hgeist2 help? Or I could publish a release from the latest master (I just don't know if it actually fixes the issue yet).

vozeldr commented 2 years ago

Yes, sorry for not providing better initial info... That folder does exist. I only encounter this problem on subsequent builds and only "occasionally". When I get this error, I delete node_modules and reinstall and it works for a random time (it could be 1 or 2 builds, it could be a day's worth of builds). I only started encountering this problem after updating to the "new parcel", but it's been a consistent issue since then.

kriszyp commented 2 years ago

So parcel/lmdb-store will work for a while, and then start throwing this error, and that point it will just continue to throw this error until you reinstall all the packages? Any chance you could see if it is resolved with the latest updates on master?

hgeist2 commented 2 years ago

I can confirm that the latest master solves the issue.

Moved project to a new VM which was based on Xbuntu 20.04, too. Got the same error with install from the repo and it was fixed by installing the current master.

kriszyp commented 2 years ago

Thank you so much for checking this out @hgeist2! I have published v1.6.12 with this fix. If anyone else is able to try that out, would love to know if fixes the other situations where this occurs.