High Memory Usage when processing HEIC file with high dimensions

[adarsh-fynd]

I'm encountering a high memory usage issue when using sharp to process a HEIC file with the density set to 300. The memory consumption seems to be significantly higher than expected, especially with HEIC files of larger dimensions. The memory usage metrics observed are:

{
  "rss": "881.21 MB -> Resident Set Size - total memory allocated for the process execution",
  "heapTotal": "7.79 MB -> total size of the allocated heap",
  "heapUsed": "6.5 MB -> actual memory used during the execution",
  "external": "2.91 MB -> V8 external memory"
}

Below is the script which I'm running:

const sharp = require("sharp");
const fs = require("fs");

const inputFilePath = "test.heif";
const outputFilePath = "output.heif";
const inputBuffer = fs.readFileSync(inputFilePath);

const processImage = async () => {
    try {
        const outputBuffer = await sharp(inputBuffer)
            .heif({ compression: "hevc" })
            .withMetadata({ density: 300 })
            .toBuffer();
        fs.writeFileSync(outputFilePath, outputBuffer);
        console.log("Image processed successfully!");
    } catch (err) {
        console.log("err => ", err);
    }
};

processImage();

Steps to Reproduce:

Use the provided script to process the HEIC file with density set to 300. Monitor the memory usage during the execution.

Expected Behavior:

The memory usage should be within reasonable limits for the given operation.

Actual Behavior:

The memory consumption is higher than expected, especially with HEIC files of larger dimensions, such as 8736px X 5856px.

Image Information:

Image Dimensions: 8736px X 5856px Format: HEIC

Environment:

Sharp Node version: 0.32.6 Node.js version: 18.16.0 Operating System: Alpine Linux

Additional Information:

I have observed that the issue is more pronounced with larger HEIC files. When testing with images of lesser dimensions, the memory usage appears to be within acceptable limits. Attached the HEIC file (8736px X 5856px) which I've used.

test.heif.zip

[lovell]

Best guess would be that this relates to the not-yet-released https://github.com/strukturag/libheif/pull/1061

Correction: this was fixed in libheif v1.17.6

[adarsh-fynd]

@lovell I'm trying to build libheif v1.17.6 from source in my dockerfile as I'm using alpine 3.17 and libheif v1.17.6 is not present in the alpine packages yet.

Here is how my dockerfile looks

RUN apk add libpng-dev libjpeg-turbo-dev libwebp-tools ffmpeg build-base libheif-dev vips-dev meson cmake

ENV LIBHEIF_VERSION=1.17.6
RUN wget https://github.com/strukturag/libheif/releases/download/v$LIBHEIF_VERSION/libheif-$LIBHEIF_VERSION.tar.gz && \
   tar -xzf libheif-$LIBHEIF_VERSION.tar.gz && \
   cd libheif-$LIBHEIF_VERSION && \
   mkdir build && \
   cd build && \
   cmake .. && \
   make && \
   make install && \
   make clean

# Download and extract the libvips source code
ENV LIBVIPS_VERSION=8.15.0
RUN wget https://github.com/libvips/libvips/releases/download/v$LIBVIPS_VERSION/vips-$LIBVIPS_VERSION.tar.xz && \
   tar xf vips-$LIBVIPS_VERSION.tar.xz && \
   cd vips-$LIBVIPS_VERSION && \
   meson setup build && \
   meson compile -C build && \
   meson install -C build

But I'm still facing the issue, can you please tell if I'm making any mistakes here?

[lovell]

v0.33.2 now available with prebuilt binaries that include libheif v1.17.6.

[adarsh-fynd]

Hi @lovell , thanks for your help here, but for some reason, I'm seeing heifsave: Unsupported compression when I install v0.33.2. I have built libvips v8.15.1 and kept node v18.17.0, but I'm still facing this issue.

Anyways, I have managed to build libheif v1.17.6 from source in my Dockerfile, but it didn't help. The memory usage is still high, and I have temporarily fixed the issue by increasing the memory size.

Another issue I'm facing is that the time taken to process the images is significantly high. It takes around 3 minutes or more to process a 50MP heic image.

Can you please let me know if we can reduce the time taken for processing?

[lovell]

You'll need to provide a complete, standalone repo that includes things like a Dockerfile, package.json, sample images and minimal code that allows someone else to reproduce.

[adarsh-fynd]

Hi @lovell , I have created this repo with all the necessary details, can you please check ? https://github.com/adarsh-fynd/heic

[lovell]

Thanks, please can you first update all the dependencies to the latest, including Alpine, Node.js, libvips and sharp.

In addition, it looks like the sample code is decoding HEIC and encoding as AVIF. Is this intentional?

[adarsh-fynd]

I tried updating to libvips v8.15.1 and sharp v0.33.2 I didn't update node js to the latest version, updated it to the v18.17.0

But I'm seeing the following error

Error: source: bad seek to 2503503
heifsave: Unsupported compression

I have updated the changes in my repo for reference.

Also, I was trying to encode as HEVC in my sample code, have updated the sample code in my repo.

[natuan62]

I have same error with sharp 0.33.3 and libvips 8.15.2, nodejs 18.20.0 @adarsh-fynd