search

denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
97.46k stars 5.37k forks source link

Error with module resolution loading `sharp` native module as dependency via `npm:` specifier #16578

Open k-right opened 2 years ago

k-right commented 2 years ago

Trying to get payload running on Deno unstable. Module resolution seems to get confused on the sharp module (specifically loading its native component).

deno run --unstable --allow-read --allow-env index.ts
error: Uncaught Error:
Something went wrong installing the "sharp" module

Cannot find module '../build/Release/sharp-darwin-x64.node'
Require stack:
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/index.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/uploads/generateFileData.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/create.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/local/create.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/local/index.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/index.js
- /Users/x/Library/Caches/deno/npm/registry.npmjs.org/payload/1.1.21/dist/index.js

Possible solutions:
- Install with verbose logging and look for errors: "npm install --ignore-scripts=false --foreground-scripts --verbose sharp"
- Install for the current darwin-x64 runtime: "npm install --platform=darwin --arch=x64 sharp"
- Consult the installation documentation: https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:34:9)
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:37:4)
    at Module._compile (deno:ext/node/02_require.js:717:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:750:12)
    at Module.load (deno:ext/node/02_require.js:634:34)
    at Function.Module._load (deno:ext/node/02_require.js:491:14)
    at Module.require (deno:ext/node/02_require.js:656:21)
    at require (deno:ext/node/02_require.js:790:18)
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:8:1)
    at Object.<anonymous> (file:///Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:425:4)

Minimal example to reproduce:

import express from 'npm:express@4.18.2'
import payload from 'npm:payload@1.1.21'

const app = express();

payload.init({
  secret: process.env.PAYLOAD_SECRET,
  mongoURL: process.env.MONGO_URL,
  express: app,
});

app.listen(process.env.PAYLOAD_PORT, process.env.PAYLOAD_ADDR, async () => {
  console.log(`listening on http://${process.env.PAYLOAD_ADDR}:${process.env.PAYLOAD_PORT}/`);
});
k-right commented 2 years ago 
cd ~/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2
npm i
deno run --unstable --allow-read --allow-env --allow-ffi index.ts
dyld: lazy symbol binding failed: Symbol not found: _pthread_jit_write_protect_supported_np
  Referenced from: /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/build/Release/../.././vendor/8.13.3/darwin-x64/lib/libvips-cpp.42.dylib
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _pthread_jit_write_protect_supported_np
  Referenced from: /Users/x/Library/Caches/deno/npm/registry.npmjs.org/sharp/0.31.2/build/Release/../.././vendor/8.13.3/darwin-x64/lib/libvips-cpp.42.dylib
  Expected in: /usr/lib/libSystem.B.dylib

[1]    48222 abort      deno run --unstable --allow-read --allow-env --allow-ffi index.ts
littledivy commented 2 years ago

@k-right It seems like a recent bug in Sharp: https://github.com/lovell/sharp/issues/3438

k-right commented 2 years ago

@littledivy Yep, looks like it.

Tried same strategies with Docker denoland/deno@latest, Node.js 16.18.1, NPM 8.19.2.

Without npm i

deno run --unstable --allow-read --allow-env --allow-ffi main.ts
✅ Granted run access to "/bin/sh".
error: Uncaught Error:
Something went wrong installing the "sharp" module

Cannot find module '../build/Release/sharp-linux-x64.node'
Require stack:
- /deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js
- /deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js
- /deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/index.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/uploads/generateFileData.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/create.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/local/create.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/collections/operations/local/index.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/index.js
- /deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/index.js

Possible solutions:
- Install with verbose logging and look for errors: "npm install --ignore-scripts=false --foreground-scripts --verbose sharp"
- Install for the current linux-x64 runtime: "npm install --platform=linux --arch=x64 sharp"
- Consult the installation documentation: https://sharp.pixelplumbing.com/install
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:34:9)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/sharp.js:37:4)
    at Module._compile (deno:ext/node/02_require.js:726:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:759:12)
    at Module.load (deno:ext/node/02_require.js:643:34)
    at Function.Module._load (deno:ext/node/02_require.js:500:14)
    at Module.require (deno:ext/node/02_require.js:665:21)
    at require (deno:ext/node/02_require.js:799:18)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:8:1)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/sharp/0.31.2/lib/constructor.js:425:4)

With npm i

deno run --unstable --allow-read --allow-env --allow-ffi --allow-run --allow-sys main.ts
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
error: Uncaught TypeError: model.prototype.$__setSchema is not a function
    at Function.compile (file:///deno-dir/npm/registry.npmjs.org/mongoose/6.5.0/lib/model.js:4937:19)
    at Mongoose._model (file:///deno-dir/npm/registry.npmjs.org/mongoose/6.5.0/lib/index.js:601:27)
    at Mongoose.model (file:///deno-dir/npm/registry.npmjs.org/mongoose/6.5.0/lib/index.js:560:27)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/preferences/model.js:27:34)
    at Object.<anonymous> (file:///deno-dir/npm/registry.npmjs.org/payload/1.1.21/dist/preferences/model.js:39:4)
    at Module._compile (deno:ext/node/02_require.js:726:36)
    at Object.Module._extensions..js (deno:ext/node/02_require.js:759:12)
    at Module.load (deno:ext/node/02_require.js:643:34)
    at Function.Module._load (deno:ext/node/02_require.js:500:14)
    at Module.require (deno:ext/node/02_require.js:665:21)

I'm thinking the former error may be caused by the lack of support for napi_add_finalizer as indicated by the latter. Let me know if there's still something to be investigated here, otherwise it's something to keep an eye on for testing if support for napi_add_finalizer is on the roadmap.

bartlomieju commented 1 year ago

Seems this is the same problem as https://github.com/denoland/deno/issues/17085

bartlomieju commented 1 year ago

@k-right it seems this example involves several other packages. Could you try to boil it down into two separate issues - one for sharp support and one for the error coming from mongoose?

hahnbeelee commented 1 year ago

I've seen the sharp error happen on old versions of Mac OS's. I know nothing about deno but just wanted to put that out there in case it helps you guys narrow down what the problem is.

https://developer.apple.com/forums/thread/707916

bartlomieju commented 1 year ago

napi_add_finalizer is now available and working. @k-right any chance you could try this again and let us know if the problem persists?

zbynekwinkler commented 1 year ago

I tried while checking

When all binary files are in place, sharp can be imported (meaning I can run deno repl and paste import sharp from 'npm:sharp'; and not get an error).

bartlomieju commented 1 year ago

@zwn by "when all binary files are in place" do you mean that you manually run postinstall script?

zbynekwinkler commented 1 year ago

@bartlomieju I just tried to import sharp and when it said some file is missing I've put it into the place where it was expected.

There were two parts to make it working:

birkskyum commented 1 year ago

Related to

shriharip commented 1 year ago

facing same issue. simple reproducible code is also the same :)

something went wrong installing the "sharp" module

Cannot find module '../build/Release/sharp-darwin-x64.node'
Require stack:
- /Users/shrihari/projects/advedaweb/node_modules/.deno/sharp@0.31.3/node_modules/sharp/lib/sharp.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/sharp@0.31.3/node_modules/sharp/lib/constructor.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/sharp@0.31.3/node_modules/sharp/lib/index.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/payload@1.15.6/node_modules/payload/dist/uploads/generateFileData.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/payload@1.15.6/node_modules/payload/dist/collections/operations/create.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/payload@1.15.6/node_modules/payload/dist/collections/requestHandlers/create.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/payload@1.15.6/node_modules/payload/dist/collections/buildEndpoints.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/payload@1.15.6/node_modules/payload/dist/collections/initHTTP.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/payload@1.15.6/node_modules/payload/dist/initHTTP.js
- /Users/shrihari/projects/advedaweb/node_modules/.deno/payload@1.15.6/node_modules/payload/dist/index.js
- /Users/shrihari/projects
birkskyum commented 10 months ago

This issue specific to sharp can be closed since sharp v0.33.