avifenc using high amount of memory

[GyllieGyllie]

We are running avifenc on AWS Lambda to convert single frame png images to a single AVIF image. It works fine for most but we have some group of images where it just exits with no feedback at all. We recompiled everything yesterday on the latest commit from the default branch.

Command ran that fails

./avifenc -j all -d 8 --minalpha 24 --maxalpha 28 --output /tmp/out/hat_300.avif /tmp/in/hat_9/hat_0.png /tmp/in/hat_9/hat_1.png /tmp/in/hat_9/hat_2.png /tmp/in/hat_9/hat_3.png /tmp/in/hat_9/hat_4.png /tmp/in/hat_9/hat_5.png /tmp/in/hat_9/hat_6.png /tmp/in/hat_9/hat_7.png /tmp/in/hat_9/hat_8.png /tmp/in/hat_9/hat_9.png

We also tried to run the same command on another EC2 instance we have where we can see this

Successfully loaded: hat_0.png
AVIF to be written: (Lossy)
 * Resolution     : 2160x2160
 * Bit Depth      : 8
 * Format         : YUV444
 * Alpha          : Not premultiplied
 * Range          : Full
 * Color Primaries: 1
 * Transfer Char. : 13
 * Matrix Coeffs. : 6
 * ICC Profile    : Absent
 * XMP Metadata   : Absent
 * Exif Metadata  : Absent
 * Transformations: None
 * Progressive    : Unavailable
Encoding with AV1 codec 'aom' speed [6], color QP [24 (Medium) <-> 26 (Medium)], alpha QP [24 (Medium) <-> 28 (Medium)], tileRowsLog2 [0], tileColsLog2 [0], 2 worker thread(s), please wait...
 * Encoding frame 1 [1/30 ts]: hat_0.png
 Killed

When finding the reason it says

 [   98.773385] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=avifenc,pid=1191,uid=1000
[   98.775138] Out of memory: Killed process 1191 (avifenc) total-vm:533556kB, anon-rss:334252kB, file-rss:8kB, shmem-rss:0kB, UID:1000 pgtables:732kB oom_score_adj:0

The instance has 0.5GB ram, on lambda we have it set to 1GB. The images themselves are each ~250KB in size and 2160x2160px

[GyllieGyllie]

Ok after more debugging it seems the issue is most likely the RAM usage. When we upgraded the EC2 instance to have 4GB we were able to convert it but the memory usage is spiking to almost 2GB. Is it normal that the memory usage is soo high for random images or is there a specific reason?

https://i.rigner.ovh/M6Ko0mkD.mp4

[jzern]

Thanks for the report and investigation. The resolution of the images are one of the main components to memory usage and with animations the encoder will keep extra image-sized buffers to use as references.

What were the sizes of the images in this case? It looks like the encoder was killed by the kernel due to the memory usage rather than avifenc seeing an allocation failure.

[GyllieGyllie]

The images were 2160x2160. The issue was indeed that the kernel killed the process as it was trying to use more memory than available.

My main concern still is why it's using so much memory for so few files while files with same size are not needed that much memory.

[aentwist]

I have a large image coming in at around 375MB, about 40,000 x 30,000. This thing spikes to 9GB of RAM usage before dying. I can't get it to work :(

avifenc -c aom --min 31 --max 31 --minalpha 31 --maxalpha 31 -s 0 s13b-map.png s13b-map.avif

[wantehchang]

Hi @aentwist, could you share s13b-map.png with me? I'll take a look.

Note that a three-channel, 40,000 x 30,000 image is 3.35GB uncompressed. So if we have two copies of the uncompressed image in memory, that's 6.7GB already. The aom_codec_encode() function called by avifenc copies the input image before compressing it.

[vrabaud]

avifenc limits decoding to AVIF_DEFAULT_IMAGE_SIZE_LIMIT, cf https://github.com/AOMediaCodec/libavif/blob/471326f276ed2fe741c1c80d9b115ba853cb5065/apps/avifenc.c#L614

libaom also limits the max_area to 1<<30: https://aomedia.googlesource.com/aom/+/6fb5bfdf30282eebf47db5ca6f0dfecedcd0853c/av1/av1_cx_iface.c#675

I tried with an image of:

• 35000 by 25000, and I get an libaom error: "Memory allocation error: Failed to allocate lag buffers"
• 28000 by 20000, and RAM usage peaks at 10GB and 30 cores are used

I you use the "-autotiling" option, less RAM will be used. For now, that limits to 8 tiles though, cf https://github.com/AOMediaCodec/libavif/blob/471326f276ed2fe741c1c80d9b115ba853cb5065/src/write.c#L1708 but we could probably remove that lock as we limit to 32 tiles above anyway.

[aentwist]

Yeah I realized this image isn't really the best fit for this issue, because such an unusually large image comes with other problems. As for the RAM, it is over my head how a 375MB PNG expands to 3.5GB, but given that, the explanation of the RAM sounds fairly reasonable. My objective was only a nice-to-have so I just gave up on the conversion. Good luck should you decide to pursue this optimization.

https://github.com/aentwist/image-of-unusual-size

[msalsbery]

As for the RAM, it is over my head how a 375MB PNG expands to 3.5GB

40000 x 30000 x 3

😉

[wantehchang]

Hi @aentwist,

I have downloaded the s13b-map.png image. You can delete it now. Thanks!

The actual image size is 38464 x 28832. In addition to the high amount of memory, it may cause integer overflows in our code. So I will need to run our code under the Undefined Behavior Sanitizer and fix the integer overflows. (As Vincent noted, our code currently imposes a maximum image size of 2^30 to avoid the known integer overflows.)

[wantehchang]

I found that the s13b-map.png image has an alpha channel, so the uncompressed image size is 38464 x 28832 x 4.

[wantehchang]

Let's use this issue for the high memory usage problem when encoding an AVIF image sequence (i.e., AVIF animation) that @GyllieGyllie originally reported.

I filed https://github.com/AOMediaCodec/libavif/issues/2271 for the support of large image size that @aentwist reported in https://github.com/AOMediaCodec/libavif/issues/1111#issuecomment-2218346811.