search

lovell / sharp

High performance Node.js image processing, the fastest module to resize JPEG, PNG, WebP, AVIF and TIFF images. Uses the libvips library.
https://sharp.pixelplumbing.com
Apache License 2.0
28.91k stars 1.29k forks source link

Support Sony RAW (.ARW) #3612

Closed uhthomas closed 1 year ago

uhthomas commented 1 year ago

Feature request

What are you trying to achieve?

I'd like to use Immich with Sony RAW, which uses sharp to process media.

https://github.com/immich-app/immich/issues/2156

Related issues link to https://github.com/libvips/libvips/issues/1304 and were closed, but it seems that it's possible to use magickload?

When you searched for similar feature requests, what did you find that might be related?

https://github.com/lovell/sharp/issues/1745

https://github.com/lovell/sharp/issues/2867

What would you expect the API to look like?

?

What alternatives have you considered?

N/A

Please provide sample image(s) that help explain this feature

N/A

lovell commented 1 year ago

Are you using a custom libvips compiled with support for ImageMagick, that itself has been compiled with support for the specific RAW format you need?

If you're not using a custom libvips then the suggestions in #2867 are worth investigating, perhaps https://github.com/justinkambic/libraw.js will do what you need.

lovell commented 1 year ago

@uhthomas Were you able to make any progress with this?

lovell commented 1 year ago

Closing due to inactivity but please feel free to reopen with more details if further help is required.

uhthomas commented 1 year ago

Sorry for the lack of activity. It looks like there is a draft PR for immich to support ARW.

https://github.com/immich-app/immich/pull/2198

The PR seems to be using the darktable cli.

Is it possible for sharp to support raw files directly @lovell?

lovell commented 1 year ago

You'll need to use a globally-installed libvips compiled with support for ImageMagick.

It looks like Immich is built on Alpine Linux, so could switch from vips to vips-magick here: https://github.com/immich-app/immich/blob/d25ddfc46bf059471a89f51c73f51617b6c9ccaf/server/Dockerfile#L26

uhthomas commented 1 year ago

You'll need to use a globally-installed libvips compiled with support for ImageMagick.

It looks like Immich is built on Alpine Linux, so could switch from vips to vips-magick here: https://github.com/immich-app/immich/blob/d25ddfc46bf059471a89f51c73f51617b6c9ccaf/server/Dockerfile#L26

This sounds great, thank you so much for the help. Assuming this is installed, are any code changes required or should it "just work"?

uhthomas commented 1 year ago

@lovell I've installed vips-magick and made sure to update to the new version of imagemagick which includes support for libraw, and consequently sony raw files.

I can see this works as expected with vips-tools, but not sharp.

/usr/src/app/upload/library/admin/2021/2021-07-22 # vips thumbnail _DSC2642.arw test.jpg 100

(vips:94): VIPS-WARNING **: 12:38:01.299: Unknown field with tag 50341 (0xc4a5) encountered

(vips:94): VIPS-WARNING **: 12:38:01.384: Unknown field with tag 50341 (0xc4a5) encountered

(vips:94): VIPS-WARNING **: 12:38:02.791: Unknown field with tag 50341 (0xc4a5) encountered

error from sharp:

immich_microservices     | [Nest] 40  - 06/05/2023, 12:31:56 PM    WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=03a24205-0e0e-48c9-abfc-5c00463827e5): Input file contains unsupported image format

Any ideas on what could be happening?

uhthomas commented 1 year ago

Looks like the error comes from here.

https://github.com/lovell/sharp/blob/93fafb0c181d8c35e8db64ba9cf7b64fae4df41e/src/common.cc#L516

uhthomas commented 1 year ago

Again, I can see imagemagick is happy and vips is also pretty happy. Seems strange for the C library not to work.

/usr/src/app/upload/library/admin/2021/2021-07-22 # vipsheader -a _DSC2642.arw

(vipsheader:129): VIPS-WARNING **: 12:41:27.105: Unknown field with tag 50341 (0xc4a5) encountered
_DSC2642.arw: 9568x6376 ushort, 3 bands, rgb16, magickload
width: 9568
height: 6376
bands: 3
format: ushort
coding: none
interpretation: rgb16
xoffset: 0
yoffset: 0
xres: 1
yres: 1
filename: _DSC2642.arw
vips-loader: magickload
magick-date:create: 2023-06-05T12:18:20+00:00
magick-date:modify: 2023-06-05T12:18:20+00:00
magick-date:timestamp: 2023-06-05T12:41:27+00:00
magick-dng:camera.model.name: ILCE-7RM4
magick-dng:create.date: 2021-07-22T18:01:26+00:00
magick-dng:exposure.time: 1/8000
magick-dng:f.number: 1.4
magick-dng:focal.length: 35.0 mm
magick-dng:focal.length.in.35mm.format: 35 mm
magick-dng:gps.altitude: 0.0 m
magick-dng:gps.latitude: 0 deg 0' 0.00" N
magick-dng:gps.longitude: 0 deg 0' 0.00" W
magick-dng:iso.setting: 100
magick-dng:lens: 35.0-0.0mm f/1.4-0.0
magick-dng:lens.f.stops: 0.00
magick-dng:lens.type:
magick-dng:make: Sony
magick-dng:max.aperture.at.max.focal: 0.0
magick-dng:max.aperture.at.min.focal: 1.4
magick-dng:max.aperture.value: 1.4
magick-dng:max.focal.length: 0.0 mm
magick-dng:min.focal.length: 35.0 mm
magick-dng:serial.number: 05074736
magick-dng:software: ILCE-7RM4 v1.20
magick-dng:wb.rb.levels: 2464.000000 1796.000000 1024.000000 1024.000000
magick-xmp:Rating: 0
xmp-data: 4097 bytes of binary data
magick-format: ARW
n-pages: 1
orientation: 1
uhthomas commented 1 year ago

Is it possible that libvips is not detecting imagemagick correctly? Seems odd given I have these following packages installed:

apk add --no-cache vips vips-cpp vips-heif vips-magick libraw-dev ffmpeg perl imagemagick vips vips-tools

https://github.com/lovell/sharp/issues/3346#issuecomment-1233095336

lovell commented 1 year ago

Did you see sharp.format? This provides runtime information about available formats.

When using sharp with a globally-installed libvips, please check the output of npm install --verbose --foreground-scripts sharp to ensure it has been detected.

uhthomas commented 1 year ago

I'm not really sure what I should be looking for from the requested info. I hope it's helpful though.

immich_server            | {
immich_server            |   jpeg: {
immich_server            |     id: 'jpeg',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server            |   },
immich_server            |   png: {
immich_server            |     id: 'png',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_server            |   webp: {
immich_server            |     id: 'webp',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_server            |   tiff: {
immich_server            |     id: 'tiff',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server            |   },
immich_server            |   magick: {
immich_server            |     id: 'magick',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   openslide: {
immich_server            |     id: 'openslide',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   dz: {
immich_server            |     id: 'dz',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_server            |   ppm: {
immich_server            |     id: 'ppm',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   fits: {
immich_server            |     id: 'fits',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   gif: {
immich_server            |     id: 'gif',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_server            |   svg: {
immich_server            |     id: 'svg',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   heif: {
immich_server            |     id: 'heif',
immich_server            |     input: { file: true, buffer: true, stream: true, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: true, stream: true, alias: [Array] }
immich_server            |   },
immich_server            |   pdf: {
immich_server            |     id: 'pdf',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   vips: {
immich_server            |     id: 'vips',
immich_server            |     input: { file: true, buffer: false, stream: false, fileSuffix: [Array] },
immich_server            |     output: { file: true, buffer: false, stream: false }
immich_server            |   },
immich_server            |   jp2k: {
immich_server            |     id: 'jp2k',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false, alias: [Array] }
immich_server            |   },
immich_server            |   jxl: {
immich_server            |     id: 'jxl',
immich_server            |     input: { file: false, buffer: false, stream: false },
immich_server            |     output: { file: false, buffer: false, stream: false }
immich_server            |   },
immich_server            |   raw: {
immich_server            |     id: 'raw',
immich_server            |     input: { file: false, buffer: true, stream: true },
immich_server            |     output: { file: false, buffer: true, stream: true }
immich_server            |   }
immich_server            | 
/usr/src/app # npm install --verbose --foreground-scripts sharp
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@9.5.1
npm info using node@v18.16.0
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/root/.npm/_logs/2023-06-05T14_35_08_361Z-
npm verb logfile /root/.npm/_logs/2023-06-05T14_35_08_361Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 357ms (cache hit)
npm verb reify failed optional dependency /usr/src/app/node_modules/fsevents
npm verb reify failed optional dependency /usr/src/app/node_modules/exiftool-vendored.exe
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-win32-x64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-linux-arm64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-linux-arm
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-darwin-x64
npm verb reify failed optional dependency /usr/src/app/node_modules/@msgpackr-extract/msgpackr-extract-darwin-arm64
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 648ms
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fcore 17ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/engine.io 26ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/socket.io-parser 29ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/request 30ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fplatform-express 10ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@openapitools%2fopenapi-generator-cli 19ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fwebsockets 28ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/socket.io 3ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/local-reverse-geocoder 2ms (cache hit)
npm http fetch GET 200 https://registry.npmjs.org/@nestjs%2fplatform-socket.io 5ms (cache hit)

up to date, audited 1025 packages in 2s

120 packages are looking for funding
  run `npm fund` for details

10 vulnerabilities (8 moderate, 2 high)

To address all issues possible, run:
  npm audit fix --force

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.
npm verb exit 0
npm info ok
lovell commented 1 year ago

immich_server | magick: { immich_server | id: 'magick', immich_server | input: { file: false, buffer: false, stream: false },

This tells me that *magick support is not available at runtime.

Perhaps run npm install --verbose --foreground-scripts sharp in a new, empty directory that is nowhere near an existing node_modules directory or package.json file, e.g. /tmp.

uhthomas commented 1 year ago

I installed sharp in /tmp and I don't think it changed much?

/usr/src/app # cd /tmp
/tmp # npm install --verbose --foreground-scripts sharp
npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using npm@9.5.1
npm info using node@v18.16.0
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/root/.npm/_logs/2023-06-12T14_30_34_457Z-
npm verb logfile /root/.npm/_logs/2023-06-12T14_30_34_457Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 323ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color 78ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-addon-api 127ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tar-fs 161ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/detect-libc 177ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tunnel-agent 174ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-get 183ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/prebuild-install 194ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/semver 208ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/pump 93ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/napi-build-utils 100ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-string 108ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/expand-template 107ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/minimist 110ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-convert 116ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/tar-stream 153ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-concat 170ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/safe-buffer 167ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/once 176ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/chownr 172ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/decompress-response 179ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/lru-cache 196ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/github-from-package 267ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/rc 311ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-abi 335ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mkdirp-classic 779ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mkdirp-classic 790ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-name 65ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/simple-swizzle 65ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/color-name 70ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/is-arrayish 56ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/end-of-stream 86ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/ini 89ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/deep-extend 162ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/strip-json-comments 161ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/wrappy 63ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/yallist 57ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/mimic-response 84ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/bl 101ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/readable-stream 99ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/inherits 204ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/fs-constants 712ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/util-deprecate 73ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/buffer 91ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/string_decoder 171ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/ieee754 81ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/base64-js 83ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-addon-api/-/node-addon-api-6.1.0.tgz 97ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/semver/-/semver-7.5.1.tgz 102ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/node-abi/-/node-abi-3.45.0.tgz 123ms (cache miss)
npm http fetch GET 200 https://registry.npmjs.org/sharp/-/sharp-0.32.1.tgz 137ms (cache miss)
npm info run sharp@0.32.1 install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)

> sharp@0.32.1 install
> (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)

sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.2/libvips-8.14.2-linuxmusl-x64.tar.brll/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
sharp: Integrity check passed for linuxmusl-x64arp@0.32.1 install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info begin Prebuild-install version 7.1.1install node_modules/sharp (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info looking for local prebuild @ prebuilds/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install info looking for cached prebuild @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install http request GET https://github.com/lovell/sharp/releases/download/v0.32.1/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install http 200 https://github.com/lovell/sharp/releases/download/v0.32.1/sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info downloading to @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz.102-57bfe0507e169.tmp
prebuild-install info renaming to @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz&& node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
prebuild-install info unpacking @ /root/.npm/_prebuilds/564a4a-sharp-v0.32.1-napi-v7-linuxmusl-x64.tar.gz
prebuild-install info unpack resolved to /tmp/node_modules/sharp/build/Release/sharp-linuxmusl-x64.node
prebuild-install info install Successfully installed prebuilt binary!
npm info run sharp@0.32.1 install { code: 0, signal: null }

added 45 packages in 5s

10 packages are looking for funding
  run `npm fund` for details
npm verb exit 0
npm info ok
lovell commented 1 year ago

sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.2/libvips-8.14.2-linuxmusl-x64.tar.br

sharp is unable to detect the globally-installed libvips. Have you installed the vips-dev package?

uhthomas commented 1 year ago

omg you solved it 😭

I created this minimal reproduction repository to work through why it wasn't working and simply adding vips-dev fixed it. I hope the repo is helpful in future as it should be an easy general purpose way to print diagnostic information on sharp runtime formats.

https://github.com/uhthomas/sharp3612

Thank you so much. I'll look at adding Sony ARW support to immich right now. You're the best!

uhthomas commented 1 year ago

Just to double check, vips-cpp isn't required by sharp right?

lovell commented 1 year ago

It is required but you don't have to explicitly install it as vips-cpp is already (correctly) a dependency of vips-dev.

uhthomas commented 1 year ago

Makes sense, thank you. Are any other packages other than vips-magick required for it to be fully functional do you know? Sharp says the format magick is supported, but it fails to actually do anything with a .ARW image.

immich_server            |   magick: {
immich_server            |     id: 'magick',
immich_server            |     input: { file: true, buffer: true, stream: true },
immich_server            |     output: { file: true, buffer: true, stream: true }
immich_server            |   },
immich_microservices     | [Nest] 29  - 06/12/2023, 7:09:29 PM    WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=6b3fb2d6-59ba-465e-809d-e319139b6127): Input file contains unsupported image format

Ignore the above, but thought it was helpful for context. Installing imagemagick-dev seems to have fixed it. Awesome!!