[API Feature Request] Installing native node add-ons using node-gyp

loris commented 2 years ago

Description

Right now, it looks like we cannot use any native node add-ons which are built using node-gyp. For instance, I tried various npm package to access sqlite database but failed: sqlite, sqlite3, better-sqlite3, etc

Steps To Reproduce

npm install and try to import in your extension any package relying on node-gyp (for instance: https://github.com/JoshuaWise/better-sqlite3)

The current behavior

You will get a nodejs error (such as Error: Could not find module root given file: "node:internal/modules/cjs/loader". Do you have apackage.jsonfile?)

The expected behavior

No error

bromanko commented 2 years ago

sqlite3 also does not work because it tries to read package.json in the module and that file isn't found when running in Raycast.

Error: package.json does not exist at /Users/bromanko/.config/raycast/extensions/package.json

        throw new Error("package.json does not exist at " + package_json_path);
              ^

Object.exports.find:pre-binding.js:18:15

---
15: 
16: exports.find = function(package_json_path,opts) {
17:    if (!existsSync(package_json_path)) {
18:         throw new Error("package.json does not exist at " + package_json_path);
19:    }
20:    var package_json = require(package_json_path);
21:    versioning.validate_config(package_json,opts);
---

node_modules/sqlite3/lib/sqlite3-binding.js:sqlite3-binding.js:3:27
__require:searchChrome.js:9:44
node_modules/sqlite3/lib/sqlite3.js:sqlite3.js:2:15
__require:searchChrome.js:9:44
Object.<anonymous>:browserHistory.tsx:4:21
Module._compile:loader:1101:14
Object.Module._extensions..js:loader:1153:10
Module.load:loader:981:32
Function.Module._load:loader:822:12

bromanko commented 2 years ago

FWIW I was able to use the WASM-based sql.js in https://github.com/raycast/extensions/pull/172

fe9lix commented 2 years ago

Hey guys, I'm flagging this as "feature" instead of bug because this is more like a known JS bundler limitation (in our case esbuild) and something that could be argued is "missing" rather than being buggy.

For now, there a workarounds such as marking certain modules as external via package.json and using other alternatives such as the mentioned wasm-based package above.

ayroblu commented 1 year ago

👋 Hey, based on this issue: https://github.com/evanw/esbuild/issues/1051 is there still anything that can't be done here? My understanding is that esbuild simply grabs all the .node files and bundles them along? Though sqlite3 fails for weird reasons, better-sqlite3 looks like it could work with this native node modules plugin approach?

I get a list of errors like:

Error: Could not locate the bindings file. Tried:
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/build/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/build/Debug/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/build/Release/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/out/Debug/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/Debug/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/out/Release/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/Release/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/build/default/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/compiled/18.10.0/darwin/arm64/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/addon-build/release/install-root/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/addon-build/debug/install-root/better_sqlite3.node
 → /Applications/Raycast.app/Contents/Resources/RaycastCommands_RaycastCommands.bundle/Contents/Resources/api/addon-build/default/install-root/better_sqlite3.node

gutenye commented 7 months ago

For now, there a workarounds such as marking certain modules as external via package.json and using other alternatives such as the mentioned wasm-based package above.

@fe9lix How to make it as external via package.json? Can you give an example?

raycast / extensions