search

lovell / sharp-libvips

Packaging scripts to prebuild libvips and its dependencies - you're probably looking for https://github.com/lovell/sharp
Apache License 2.0
169 stars 95 forks source link

Thread oversubscription on musl-based Linux #229

Open kleisauke opened 3 months ago

kleisauke commented 3 months ago

Reproducer:

$ mkdir sharp-alpine && cd sharp-alpine
$ docker run -v $(pwd):/app -w /app -it alpine:latest sh -c "apk add npm && npm install sharp && VIPS_PROGRESS=1 node -e \"require('sharp')({ create: { width: 1000, height: 1000, channels: 3, background: 'red' } }).toFile('x.png')\""
...
(process:1): VIPS-WARNING **: 18:05:50.814: threads clipped to 1024
sharp temp-17: 1000 x 1000 pixels, 1024 threads, 1000 x 16 tiles, 16896 lines in buffer
sharp temp-17: done in 0.0117s
$ nproc
24

Regressed since commit https://github.com/lovell/sharp-libvips/commit/eb6fedeea4dacbc214cf978a5ba75657295ac274 (sharp v0.33.3) and fixed in GLib with commit https://github.com/GNOME/glib/commit/cc25486b233ada380ac8452f47f5fb35536888f4 (https://gitlab.gnome.org/GNOME/glib/-/merge_requests/3979).

lovell commented 3 months ago

Ouch, that's probably going to hurt performance. I guess it might be worth adding a hard-coded value for musl-based users as a workaround for now (like sharp already does for non-jemalloc glibc users).

kleisauke commented 3 months ago

It looks like Alpine has the same issue: https://gitlab.alpinelinux.org/alpine/aports/-/issues/15972

GLib 2.80.1 will probably be released soon. Setting the VIPS_CONCURRENCY environment variable is a viable workaround for now.