haraldk / TwelveMonkeys

TwelveMonkeys ImageIO: Additional plug-ins and extensions for Java's ImageIO
https://haraldk.github.io/TwelveMonkeys/
BSD 3-Clause "New" or "Revised" License
1.83k stars 308 forks source link

WebP: OOM for test file #730

Open BigMichi1 opened 1 year ago

BigMichi1 commented 1 year ago

Describe the bug we have a sample webp file which causes an OOM and we want to understand if this is something we need to expect (reading webp files need a lot of memory) or if there is something not correct in the image reader.

Version information

  1. The version of the TwelveMonkeys ImageIO library in use. 3.9.4

  2. The exact output of java --version (or java -version for older Java releases). For example:

    openjdk version "17.0.5" 2022-10-18
    OpenJDK Runtime Environment Temurin-17.0.5+8 (build 17.0.5+8)
    OpenJDK 64-Bit Server VM Temurin-17.0.5+8 (build 17.0.5+8, mixed mode, sharing)
  3. Extra information about OS version, server version, standalone program or web application packaging, executable wrapper, etc. Please state exact version numbers where applicable.

To Reproduce Steps to reproduce the behavior:

  1. Compile the below sample code
  2. Download the sample image file
  3. Run the code with the sample file and Xmx set to 1024m or lower
  4. See error

Expected behavior should work with Xmx lower than 1280m

Example code

    @Test
    public void triggerOOM() throws IOException {
        try (final var file = new ClassPathResource("sample-webp-file-for-testing.webp").getInputStream()) {
            final var image = ImageIO.read(file);
            assertThat(image).isNotNull();
        }
    }

Less is more. Don't add your entire project, only the code required to reproduce the problem. 😀

Sample file(s) sample-webp-file-for-testing.zip

Stak trace

  java.lang.OutOfMemoryError: Java heap space
    at com.twelvemonkeys.imageio.plugins.webp.vp8.MacroBlock.predictUV(MacroBlock.java:492)
    at com.twelvemonkeys.imageio.plugins.webp.vp8.MacroBlock.dequantMacroBlock(MacroBlock.java:211)
    at com.twelvemonkeys.imageio.plugins.webp.vp8.VP8Frame.decodeMacroBlockRow(VP8Frame.java:366)
    at com.twelvemonkeys.imageio.plugins.webp.vp8.VP8Frame.decode(VP8Frame.java:346)
    at com.twelvemonkeys.imageio.plugins.webp.WebPImageReader.readVP8(WebPImageReader.java:657)
    at com.twelvemonkeys.imageio.plugins.webp.WebPImageReader.readVP8Extended(WebPImageReader.java:484)
    at com.twelvemonkeys.imageio.plugins.webp.WebPImageReader.readVP8Extended(WebPImageReader.java:463)
    at com.twelvemonkeys.imageio.plugins.webp.WebPImageReader.read(WebPImageReader.java:441)
    at java.desktop/javax.imageio.ImageIO.read(ImageIO.java:1470)
    at java.desktop/javax.imageio.ImageIO.read(ImageIO.java:1365)
haraldk commented 1 year ago

Thanks for reporting!

While your WebP file is rather large (8000 x 6000 x 3 ~= 140 MB in memory), I have to agree that the memory requirements for decoding seem a bit unreasonable...

I didn't write this code myself, so I don't know it with enough detail to say much about why the requirements are this high, or what can be done to optimize the memory usage. Any insights or suggestions at optimizations are very welcome!

mcpierce commented 9 months ago

I'm having a similar issue with running things on a Raspberry Pi box. My project's unit tests fail, but they pass on other platforms (MacOS, Windows, Linux).