PNG with zero-length EXIF: reached chunk/cache limits

jkh19 commented 1 year ago

Possible bug

Is this a possible bug in a feature of sharp, unrelated to installation?

[x] Running npm install sharp completes without error.
[x] Running node -e "require('sharp')" completes without error.

If you cannot confirm both of these, please open an installation issue instead.

Are you using the latest version of sharp?

[x] I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.

If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.

What is the output of running `npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp`?

  System:
    OS: Windows 10 10.0.22621
    CPU: (12) ia32 Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
    Memory: 3.55 GB / 31.75 GB
  Binaries:
    Node: 14.16.1 - C:\Program Files (x86)\nodejs\node.EXE
    npm: 6.14.12 - C:\Program Files (x86)\nodejs\npm.CMD
  npmPackages:
    sharp: ^0.32.0 => 0.32.0

What are the steps to reproduce?

When attempting to use certain PNG images in various operations (e.g., retrieve metadata, resize), the following error is received: Error: Input file has corrupt header: pngload: reached chunk/cache limits

I came across an issue with a similar error message, but setting the unlimited option flag to true does not seem to have any impact.

What is the expected behaviour?

The associated operation (retrieve metadata, resize) should succeed with these images, unless there is something inherently wrong with these files, though I have encountered this issue with over 100 files thus far.

Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this problem

Running node .\testPng.js with the following results in An error occurred during processing: Error: Input file has corrupt header: pngload: reached chunk/cache limits. Note that github.png is provided below as part of this issue.

testPng.js:

const Sharp = require("sharp");

async function getMetadata() {
  try {
    const metadata = await Sharp("github.png", {unlimited: true}).metadata();
    console.log(metadata);
  } catch (error) {
    console.log(`An error occurred during processing: ${error}`);
  }
}

getMetadata();

Please provide sample image(s) that help explain this problem

github.png:

response4

Additional analysis of png file

> pngchunks github.png
Chunk: Data Length 13 (max 2147483647), Type 1380206665 [IHDR]
  Critical, public, PNG 1.2 compliant, unsafe to copy
  IHDR Width: 474
  IHDR Height: 203
  IHDR Bitdepth: 8
  IHDR Colortype: 6
  IHDR Compression: 0
  IHDR Filter: 0
  IHDR Interlace: 0
  IHDR Compression algorithm is Deflate
  IHDR Filter method is type zero (None, Sub, Up, Average, Paeth)
  IHDR Interlacing is disabled
  Chunk CRC: -921087451
Chunk: Data Length 9 (max 2147483647), Type 1935231088 [pHYs]
  Ancillary, public, PNG 1.2 compliant, safe to copy
  ... Unknown chunk type
  Chunk CRC: -1792340453
Chunk: Data Length 0 (max 2147483647), Type 1716082789 [eXIf]
  Ancillary, public, PNG 1.2 compliant, safe to copy
  ... Unknown chunk type
  Chunk CRC: -898001148
Chunk: Data Length 65535 (max 2147483647), Type 1413563465 [IDAT]
  Critical, public, PNG 1.2 compliant, unsafe to copy
  IDAT contains image data
  Chunk CRC: 184233720
Chunk: Data Length 1462 (max 2147483647), Type 1413563465 [IDAT]
  Critical, public, PNG 1.2 compliant, unsafe to copy
  IDAT contains image data
  Chunk CRC: -1722838905
Chunk: Data Length 0 (max 2147483647), Type 1145980233 [IEND]
  Critical, public, PNG 1.2 compliant, unsafe to copy
  IEND contains no data
  Chunk CRC: -1371381630

> identify -verbose github.png
Image: github.png
  Format: PNG (Portable Network Graphics)
  Mime type: image/png
  Class: DirectClass
  Geometry: 474x203+0+0
  Resolution: 37.8x37.8
  Print size: 12.5397x5.37037
  Units: PixelsPerCentimeter
  Type: TrueColorAlpha
  Endianess: Undefined
  Colorspace: sRGB
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
    alpha: 1-bit
  Channel statistics:
    Pixels: 96222
    Red:
      min: 0 (0)
      max: 255 (1)
      mean: 203.356 (0.797476)
      standard deviation: 80.0715 (0.314006)
      kurtosis: 0.68982
      skewness: -1.49184
      entropy: 0.667276
    Green:
      min: 0 (0)
      max: 255 (1)
      mean: 206.761 (0.810826)
      standard deviation: 79.2277 (0.310697)
      kurtosis: 0.793384
      skewness: -1.51669
      entropy: 0.645526
    Blue:
      min: 0 (0)
      max: 255 (1)
      mean: 208.438 (0.817405)
      standard deviation: 78.8095 (0.309057)
      kurtosis: 0.841869
      skewness: -1.52802
      entropy: 0.600285
    Alpha:
      min: 255 (1)
      max: 255 (1)
      mean: 255 (1)
      standard deviation: 0 (0)
      kurtosis: 0
      skewness: 0
      entropy: 0
  Image statistics:
    Overall:
      min: 0 (0)
      max: 255 (1)
      mean: 154.639 (0.606427)
      standard deviation: 68.7376 (0.269559)
      kurtosis: 6.81664
      skewness: -2.02606
      entropy: 0.478272
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Background color: white
  Border color: srgba(223,223,223,1)
  Matte color: grey74
  Transparent color: none
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 474x203+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: Zip
  Orientation: Undefined
  Properties:
    date:create: 2023-04-19T23:25:28-04:00
    date:modify: 2023-04-19T23:25:27-04:00
    png:IHDR.bit-depth-orig: 8
    png:IHDR.bit_depth: 8
    png:IHDR.color-type-orig: 6
    png:IHDR.color_type: 6 (RGBA)
    png:IHDR.interlace_method: 0 (Not interlaced)
    png:IHDR.width,height: 474, 203
    png:pHYs: x_res=3780, y_res=3780, units=1
    png:sRGB: intent=0 (Perceptual Intent)
    signature: 59f9052a04db41e378cd7aefb15703cf0d39f23bf56a6ca8a537298b4e662c1a
  Artifacts:
    filename: github.png
    verbose: true
  Tainted: False
  Filesize: 67.1KB
  Number pixels: 96.2K
  Pixels per second: 9.622MB
  User time: 0.000u
  Elapsed time: 0:01.010
  Version: ImageMagick 6.9.7-4 Q16 x86_64 20170114 http://www.imagemagick.org
identify-im6.q16: eXIf: too short `github.png' @ warning/png.c/MagickPNGWarningHandler/1654.

Note the warning about too short eXIf data in the last line of the identify output, if that is suspect?

lovell commented 1 year ago

Yes, this PNG is corrupt. If present, an EXIF chunk has to be at least 4 bytes (to determine endianness) but this image defines a zero-length chunk. It's much safer for a decoder to bail out at this point rather than try to keep going.

Chunk: Data Length 0 (max 2147483647), Type 1716082789 [eXIf]

It might be worth creating a new issue for this specific image upstream at https://github.com/randy408/libspng and link it to https://github.com/randy408/libspng/issues/14 as it may fall under into the category of conformance vs compatibility.

lovell commented 1 year ago

Upstream issue at https://github.com/randy408/libspng/issues/247

randy408 commented 1 year ago

A fix is included in the latest v0.7.4 release.

jkh19 commented 1 year ago

@lovell - With the upstream issue addressed, would it be possible to please include this new version of libspng in a future release?

I am just trying to weigh the option of investigating workarounds in the meantime.

Thanks!

lovell commented 1 year ago

@randy408 Thank you!

@jkh19 Yes. If taking advantage of the upstream fix is urgent for you, please remember that sharp will always look for and use a globally-installed libvips (and its dependencies) if available, so you may want to consider that route.

jkh19 commented 1 year ago

@lovell Thanks for the response.

Please forgive my ignorance, but I have been kind of floundering with your recommendation (which I have been trying to pursue as it is becoming a more pressing issue for my needs).

From my understanding, Sharp uses libvips, which uses libspng. So if I get my environment updated with the latest libspng, would it end up getting picked up by libvips (and in turn Sharp)?

I am running this inside an AWS Lambda function, but I have generally been unsuccessful in trying to even get the latest libspng available in a standalone environment (since it at least doesn't seem to be available as a package to install via apt-get or npm, and attempts to build via meson or cmake seem to require later versions than I have available in my Ubuntu running on WSL).

Any insight on the best approach to getting this globally-installed libvips option in place would be greatly appreciated (or a 0.33.0 release 😄 ...unless that ends up being dependent on libvips being updated first).

lovell commented 1 year ago

@jkh19 For Lambda, perhaps look at using a layer e.g. something based on https://github.com/zoellner/sharp-heic-lambda-layer however waiting may be easier.

jkh19 commented 1 year ago

@lovell Great, thanks - I did consider that it could end up being a lambda layer, but I was struggling with the equivalent commands that would get libspng installed and available (I assume it'd be similar to the Makefile in the repo you linked, where it grabs the latest .tar.gz file, and then uses cmake or meson based on the libspng docs?).

Do you have any guess on the 0.33.0 release timing?

In the mean time, I'll take a look at the layer approach so thanks for the tip.

lovell commented 1 year ago

The prebuilt binaries provided by sharp v0.32.4 include libspng v0.7.4, thanks all for reporting/fixing.

lovell / sharp