File permissions issue in 0.26.0

[danielSONCCO]

A project stopped working in my Linux PC after upgrading sharp from 0.25.4 to 0.26.0 It's still crashing using 0.26.1

It looks linked to the file permissions of libraries used by vendor/8.10.0/libvips.so but the current error handling of sharp isn't giving enough information. Here I describe a simplified project to reproduce the issue:

Simplified project

- Start from an empty project in Linux (not nodejs/npm/node-gyp/g++/python, verify with commands like `whereis node-gyp`) - Remove `~/,cache` `~/.npm` - Install g++ 7.5.0, python 2.7.17 and python 3.6.9 - Download npm in a folder `folderName` from https://nodejs.org/en/ - Install npm@6.14.8 `sudo folderName/bin/npm i npm@6.14.8 -g` - With the new npm, install yarn@1.22.10 n@6.7.0 globally - Install the last stable version of nodejs `sudo n stable` (12.18.4 in my pc) - In a new folder, write this project: - package.json (see below) - main.js (see below) - foo.c (see below) - `yarn install` - `node main.js` (it should end normally but it's crashing, using `sudo` it's working but it shouldn't be needed) Using breakpoints in `sharp` I see that it fails also with `require("./node_modules/sharp/build/Release/sharp.node");` Using breakpoints in `require` I see that it fails aso with `process.dlopen({ exports: {} }, "/home/daniel/360/simplified_case_sharp/node_modules/sharp/build/Release/sharp.node");` Purpose of `foo.c` is to investigate why `dlopen` is crashing opening the dynamic library `sharp.node` - `gcc -o foo -g foo.c -ldl && ./foo` is returning the `error.log` (see below). I consider it weird because of the next two steps - Go to `node_modules/sharp` and we run `npm run install` - Go to previous folder, and run previous command `gcc -o foo -g foo.c -ldl && ./foo`. It's crashing but it shouldn't. Using gdb I see the `backtrace.log` (see below).

package.json

```json { "name": "untitled", "version": "1.0.0", "private": true, "license": "UNLICENSED", "dependencies": { "sharp": "^0.26.1" } } ```

main.js

```javascript require("sharp"); ```

foo.c

```c #include #include #include int main(int argc, char **argv) { void *handle; handle = dlopen ("./node_modules/sharp/build/Release/sharp.node", RTLD_LAZY); if (!handle) { puts("error:"); puts(dlerror()); return 0; } dlclose(handle); } ```

error.log

``` error: ./node_modules/sharp/build/Release/sharp.node: undefined symbol: napi_create_error ```

backtrace.log

``` (gdb) bt #0 0x0000000000000000 in ?? () #1 0x00007ffff660af77 in _g_param_type_init () from /home/daniel/360/simplified_case_sharp/./node_modules/sharp/build/Release/../../vendor/8.10.0/lib/libvips.so.42 #2 0x00007ffff5dcd222 in ?? () from /home/daniel/360/simplified_case_sharp/./node_modules/sharp/build/Release/../../vendor/8.10.0/lib/libvips.so.42 #3 0x00007ffff7de5783 in call_init (env=0x7fffffffde88, argv=0x7fffffffde78, argc=1, l=) at dl-init.c:72 #4 _dl_init (main_map=main_map@entry=0x5555557562d0, argc=1, argv=0x7fffffffde78, env=0x7fffffffde88) at dl-init.c:119 #5 0x00007ffff7dea24f in dl_open_worker (a=a@entry=0x7fffffffdb00) at dl-open.c:522 #6 0x00007ffff774051f in __GI__dl_catch_exception (exception=0x7fffffffdae0, operate=0x7ffff7de9e10 , args=0x7fffffffdb00) at dl-error-skeleton.c:196 #7 0x00007ffff7de981a in _dl_open (file=0x555555554838 "./node_modules/sharp/build/Release/sharp.node", mode=-2147483647, caller_dlopen=0x55555555476a , nsid=, argc=1, argv=, env=0x7fffffffde88) at dl-open.c:605 #8 0x00007ffff79caf96 in dlopen_doit (a=a@entry=0x7fffffffdd30) at dlopen.c:66 #9 0x00007ffff774051f in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffdcd0, operate=0x7ffff79caf40 , args=0x7fffffffdd30) at dl-error-skeleton.c:196 #10 0x00007ffff77405af in __GI__dl_catch_error (objname=0x555555756270, errstring=0x555555756278, mallocedp=0x555555756268, operate=, args=) at dl-error-skeleton.c:215 #11 0x00007ffff79cb745 in _dlerror_run (operate=operate@entry=0x7ffff79caf40 , args=args@entry=0x7fffffffdd30) at dlerror.c:162 #12 0x00007ffff79cb051 in __dlopen (file=, mode=) at dlopen.c:87 #13 0x000055555555476a in main (argc=1, argv=0x7fffffffde78) at foo.c:8 ```

The need of sudo node main.js suggest issues with file permissions. The backtrace.log suggests that it's libvips.so.42 that is crashing.

Questions

• If you arrive to see error.log, could you explain it?
• If you don't reproduce the crashing with foo.c, could you give me hints to investigate further?

Output of running npx envinfo --binaries --system?

  System:
    OS: Linux 4.15 Ubuntu 18.04.5 LTS (Bionic Beaver)
    CPU: (8) x64 Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
    Memory: 2.70 GB / 15.42 GB
    Container: Yes
    Shell: 4.4.20 - /bin/bash
  Binaries:
    Node: 12.18.4 - /usr/local/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 6.14.8 - /usr/local/bin/npm

[lovell]

The "undefined symbol: napi_create_error" error when running foo.c is because sharp requires symbols from node itself but the runtime linker hasn't been told about them (the precompiled binaries are designed to be opened by the node process rather than third-party C code), so this is a bit of a red herring.

The prebuilt binaries provided by v0.26.0+ are statically linked, which might be exposing an existing permissions problem with your set up.

Perhaps the use of sudo in sudo n stable is causing this? Tools such as n are generally designed to avoid the use of root/sudo (a good thing).

A backtrace from node crashing (rather than foo) might provide more relevant information.

[danielSONCCO]

@lovell Thank you very much for your answer.

Perhaps the use of sudo in sudo n stable is causing this?

n stable isn't working because it tries to make the directory /usr/local/n/versions/node/12.18.4 and rights of /usr/local are 755. That's the usual rights right?

A backtrace from node main.js crashing might provide more relevant information

Do you know a way to get a detailed backtrace? Using segfault-handler I obtained:

PID 6747 received SIGSEGV for address: 0x0
/home/daniel/360/simplified_case_sharp/node_modules/segfault-handler/build/Release/segfault-handler.node(+0x2d66)[0x7f6ad99f9d66]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x128a0)[0x7f6adc8048a0]

[lovell]

n stable isn't working because it tries to make the directory /usr/local/n/versions/node/12.18.4 and rights of /usr/local are 755. That's the usual rights right?

Please see https://github.com/tj/n#installation for avoiding the need to use sudo. https://github.com/tj/n/pull/593 might also be relevant.

Do any of these problems occur if you install node via a means other than n?

[danielSONCCO]

@lovell

Do any of these problems occur if you install node via a means other than n?

Yes. Now I followed those steps and it's still crashing

• remove node/npm/yarn/n
• Install node12/npm from Ubuntu's package-manager https://github.com/nodesource/distributions/blob/master/README.md
• Install yarn: sudo npm i yarn -g. sudo is needed because it writes /usr/lib/node_modules/yarn
• Rebuild project:
  • clean project: rm -rf node_modules yarn.lock
  • yarn install
  • node main.js

[lovell]

Does this occur if you use npm install rather than yarn install?

Please can you provide a Dockerfile with the exact environment and configuration that causes this to allow someone else to reproduce it.

[danielSONCCO]

@lovell

Does this occur if you use npm install rather than yarn install?

Yes.

Please can you provide a Dockerfile with the exact environment and configuration that causes this to allow someone else to reproduce it.

I tried to reproduce it in a Dockerfile but the Dockerfile is succeeding.

Dockerfile

```Dockerfile FROM ubuntu:18.04 RUN apt-get update && apt-get install curl -y RUN curl -sL https://deb.nodesource.com/setup_12.x | bash - RUN apt-get install build-essential -y RUN apt-get install nodejs -y RUN useradd -m daniel RUN echo daniel:fake-password && chpasswd USER daniel RUN mkdir -p /home/daniel/short-project WORKDIR /home/daniel/short-project COPY main.js . COPY package.json . RUN npm install RUN node main.js ```

I also tried to reset my setup to ensure I'm aligned with that Dockerfile but I still see the issue. But I found a way to use gdb with node main.js and now I see this backtrace.log

backtrace.log

``` (gdb) bt #0 0x0000000000000000 in ?? () #1 0x00007fffdf253f77 in _g_param_type_init () from /home/daniel/360/simplified_case_sharp/node_modules/sharp/build/Release/../../vendor/8.10.0/lib/libvips.so.42 #2 0x00007fffdea16222 in ?? () from /home/daniel/360/simplified_case_sharp/node_modules/sharp/build/Release/../../vendor/8.10.0/lib/libvips.so.42 #3 0x00007ffff7de5783 in call_init (env=0x7fffffffdec0, argv=0x7fffffffdea8, argc=2, l=) at dl-init.c:72 #4 _dl_init (main_map=main_map@entry=0x2e18270, argc=2, argv=0x7fffffffdea8, env=0x7fffffffdec0) at dl-init.c:119 #5 0x00007ffff7dea24f in dl_open_worker (a=a@entry=0x7fffffffb8f0) at dl-open.c:522 #6 0x00007ffff6b8851f in __GI__dl_catch_exception (exception=0x7fffffffb8d0, operate=0x7ffff7de9e10 , args=0x7fffffffb8f0) at dl-error-skeleton.c:196 #7 0x00007ffff7de981a in _dl_open (file=0x2e2ff08 "/home/daniel/360/simplified_case_sharp/node_modules/sharp/build/Release/sharp.node", mode=-2147483647, caller_dlopen=0x9e1311 const&)::{lambda(node::binding::DLib*)#1}::operator()(node::binding::DLib*) const+65>, nsid=, argc=2, argv=, env=0x7fffffffdec0) at dl-open.c:605 #8 0x00007ffff79caf96 in dlopen_doit (a=a@entry=0x7fffffffbb20) at dlopen.c:66 #9 0x00007ffff6b8851f in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffbac0, operate=0x7ffff79caf40 , args=0x7fffffffbb20) at dl-error-skeleton.c:196 #10 0x00007ffff6b885af in __GI__dl_catch_error (objname=0x2e4a700, errstring=0x2e4a708, mallocedp=0x2e4a6f8, operate=, args=) at dl-error-skeleton.c:215 #11 0x00007ffff79cb745 in _dlerror_run (operate=operate@entry=0x7ffff79caf40 , args=args@entry=0x7fffffffbb20) at dlerror.c:162 #12 0x00007ffff79cb051 in __dlopen (file=, mode=) at dlopen.c:87 #13 0x00000000009e1311 in node::binding::DLOpen(v8::FunctionCallbackInfo const&)::{lambda(node::binding::DLib*)#1}::operator()(node::binding::DLib*) const () #14 0x00000000009e07e2 in node::binding::DLOpen(v8::FunctionCallbackInfo const&) () #15 0x0000000000bf02a9 in v8::internal::MaybeHandle v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::BuiltinArguments) () #16 0x0000000000bf2097 in v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) () #17 0x00000000013cf199 in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit () at ../../deps/v8/../../deps/v8/src/builtins/base.tq:3418 #18 0x0000000001354a44 in Builtins_InterpreterEntryTrampoline () at ../../deps/v8/../../deps/v8/src/builtins/base.tq:357 #19 0x00003d7a781004b1 in ?? () #20 0x0000320328af5a21 in ?? () #21 0x0000000700000000 in ?? () #22 0x00003d7a78100591 in ?? () #23 0x00001497f0df2b11 in ?? () #24 0x00001497f0df3391 in ?? () #25 0x0000298499200a01 in ?? () ```

I'd be glad if you can provide me a libvips.so.42 compiled with -g. Like that I can see which of my files has the issue with permission rights.

[lovell]

The scripts and environments that build the libvips binaries provided by sharp are all open source so you can adapt them as you need - see https://github.com/lovell/sharp-libvips

[danielSONCCO]

@lovell Thanks ! It fixed my issue.

I executed sudo ./build.sh 8.10.0 linux-x64 on master Then I replaced my cached file ~/.npm/_libvips/libvips-8.10.0-linux-x64.tar.br by the generated one Now sharp is working normally

Note: executing build.sh I found two details:

• line VERSION_SVG=2.50.0 is out of date, I fixed replacing 2.50.0 by 2.50.1
• line curl -Ls https://download.osgeo.org/libtiff/tiff-${VERSION_TIFF}.tar.gz | tar xzC ${DEPS}/tiff --strip-components=1 isn't working because of expired certificates of the url. I fixed replacing -Ls by -Lsk

[lovell]

I'm still unsure what was wrong with your set up, but glad you got it working.

I've notified OSGeo about their cert - https://twitter.com/lovell/status/1312422320355577856

[danielSONCCO]

@lovell I discarded https://github.com/lovell/sharp-libvips/commit/942a585783676d0e068bf533eaaddb1d3cab1d7f because the tarball without heif is also working with my scenario.

So I think it's https://github.com/lovell/sharp-libvips/commit/8c8c29c90657519a7a62b2dd4c346ce800e2b961 that fixed the issue because as I seen in https://github.com/kleisauke/net-vips/issues/90, without -Bsymbolic-functions we may use the Unity's glib.

In my Ubuntu I have /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.5600.4. So probably there was an issue allocating memory from that place.