heif_enc generate corrupted avif files when the resolution of input image is large

[leodream]

When converting large resolution images to avif, heif-enc exits successfully. But the generated avif file cannot be opened. Test image -- https://commons.wikimedia.org/wiki/File:LARGE_elevation.jpg 10800x5400

-> heif-enc -A LARGE_elevation.jpg -o t.avif
-> echo $?
0
-> identify -verbose t.avif
Error parsing OBU data
identify: Decoder plugin generated an error: Unspecified (7.0) `t.avif' @ error/heic.c/IsHEIFSuccess/139.

It looks like the issue appears after upgrading libheif from 1.14.2-5.1 to 1.14.2-8.3.

[bigcat88]

Maybe it is related to Issue 3307: picture with size equal to 4096x2304 cannot be encoded with a single tile

[farindk]

I just tried to encode the picture myself (using AOM v3.3.0) and I could decode it fine with heif-convert. Hence, it could be either a bug in your specific version of AOM or of identify. Can you attach your encoded file so that I can check whether the file is broken or the identify command? Also send the output of heif-enc -A --list-encoders.

BTW: identify also works on that file for me:

$ identify enc.avif 
enc.avif HEIC 10800x5400 10800x5400+0+0 8-bit YCbCr 0.020u 0:00.009

but it incorrectly classifies the file as HEIC (ImageMagick 6.9.11):

$ identify -verbose enc.avif 
Image:
  Filename: enc.avif
  Format: HEIC (Apple High efficiency Image Format)
  Mime type: image/x-heic

[leodream]

Thanks for the info. Here is the generated avif file -- t.avif.gz My Imagemagick version is 7.1.0.61-1.1, libaom3 3.6.0-1.1

[farindk]

I can confirm that your test file t.avif cannot be decoded. Dav1d and AOM both complain about a bitstream error. That looks like the encoder you were using was broken.

I have encoded the JPEG image with AOM, SVT, and Rav1e and all worked well. These are the encoder versions that I used:

$ heif-enc -A --list-encoders
HEIC encoders:
- x265 = x265 HEVC encoder (3.5+1-f0c1022b6) [default]
AVIF encoders:
- aom = AOMedia Project AV1 Encoder v3.3.0 [default]
- svt = SVT-AV1 encoder v1.2.1
- rav1e = Rav1e encoder

Which versions are you using?

[leodream]

❯ heif-enc -A --list-encoders
Encoders (first is default):
- aom = AOMedia Project AV1 Encoder v3.6.0
- svt = SVT-AV1 encoder v1.4.1
- rav1e = Rav1e encoder

[biggianteye]

Thanks for the info. Here is the generated avif file -- t.avif.gz My Imagemagick version is 7.1.0.61-1.1, libaom3 3.6.0-1.1

An indication that the generation has failed that hasn't been pointed out yet is that the image is completely black. One of my colleagues has been seeing this error message and behaviour as well and is currently trying to track down what specific conditions are causing the failure. It's a shame that the generation process returns a success exit code despite actually producing a broken image.

[farindk]

I tried again to decode your image t.avif with libaom 3.6.0 and it worked:

Decoding fails with dav1d and libaom 3.3.0. Since dav1d is currently the default, if installed, you have to decode it with heif-convert t.avif out.jpg -d aom, and you need to have libaom 3.6.0 installed (you can check with heif-convert --list-decoders).

Everything works fine with smaller images (your image has size 10800x5400). I have tried a couple of sizes. 5000x5000 still works for me while 6000x6000 doesn't.

Maybe @wantehchang can provide some insight what changes between libaom 3.3.0 and libaom 3.6.0 enabled the larger sizes and what the actual limits are.

[homm]

By the way, macOS 13.4.1 opens t.avif as a completely black plane.

[leodream]

I tried again to decode your image t.avif with libaom 3.6.0 and it worked:

Decoding fails with dav1d and libaom 3.3.0. Since dav1d is currently the default, if installed, you have to decode it with heif-convert t.avif out.jpg -d aom, and you need to have libaom 3.6.0 installed (you can check with heif-convert --list-decoders).

Everything works fine with smaller images (your image has size 10800x5400). I have tried a couple of sizes. 5000x5000 still works for me while 6000x6000 doesn't.

Maybe @wantehchang can provide some insight what changes between libaom 3.3.0 and libaom 3.6.0 enabled the larger sizes and what the actual limits are.

Re-generated the avif and it worked. Thanks. (libaom v3.6.1)

[wantehchang]

This is caused by a bug in libaom v3.6.0, reported in https://crbug.com/aomedia/2871#c12. The bug is fixed in libaom v3.6.1.

Note: The bug fix consists of the following two changelists: https://aomedia-review.googlesource.com/c/aom/+/174421 https://aomedia-review.googlesource.com/c/aom/+/174841

I will see if we can work around this libaom v3.6.0 bug.

[wantehchang]

It is not possible to work around this libaom v3.6.0 bug. However, we can detect it and cause the image encoding process to fail. I demonstrated the workaround in the libavif pull request https://github.com/AOMediaCodec/libavif/pull/1467, but I am not sure if we should do this.

The best solution is to upgrade to libaom v3.6.1 or later.

[farindk]

@wantehchang Thank you for the background on this bug. If we may, I would like to copy this bug detection so that we can ask the user to upgrade to v3.6.1 instead of silently producing corrupt files.

[bradh]

Perhaps we should just make the cmake test require at least 3.6.1?

[bigcat88]

Perhaps we should just make the cmake test require at least 3.6.1?

please do not do that, many builds are done with 3.3.0 and 3.5.0

[bradh]

Perhaps we should just make the cmake test require at least 3.6.1?

please do not do that, many builds are done with 3.3.0 and 3.5.0

This is the point of my suggestion: those versions have bugs, and those bugs are assigned to libheif.

[homm]

Perhaps we should just make the cmake test require at least 3.6.1?

Not "at least 3.6.1", but anything except 3.6.0?

[farindk]

The test cannot be made at build time as the user can use any dynamically linked version. We have to check this a runtime.

[farindk]

I have added a simple bug detection heuristic.

@wantehchang The bug detection logic in https://github.com/AOMediaCodec/libavif/pull/1467 seems to be too strict. According to that logic, an image exceeding 8192x4352 pixels should fail to encode, but I successfully encoded several images larger than that. E.g. 8200x4400 still works. It didn't work anymore for 8200x4800.

Thus, I have added a simpler, stricter heuristic that sets the limit to 8192x4352, which should be large enough for most cases anyway.

[farindk]

Also the limits for width and height in https://github.com/AOMediaCodec/libavif/pull/1467 seem to be wrong. The maximum width should be 16384, but I encoded width=16385 successfully.

[wantehchang]

This libaom bug was introduced in libaom 3.6.0 and fixed in libaom 3.6.1. So libaom 3.6.0 is the only libaom version with this bug.

Regarding the bug detection logic in https://github.com/AOMediaCodec/libavif/pull/1467 being too strict: I looked into it. I found that it is just a necessary condition. To get a necessary and sufficient condition, we also need to emulate the av1_get_tile_limits() function in av1/common/tile_common.c: https://aomedia-review.googlesource.com/c/aom/+/174421

and detect whether the use of MAX_TILE_AREA_LEVEL_7_AND_ABOVE actually results in a difference (specifically, a different value of tiles->min_log2). It seems difficult to emulate that, so I am afraid that we will need to be content with a necessary condition. We can add a comment to point out that the bug detection condition is just a necessary condition.

As for why you encoded width=16385 successfully, I will need to know what the height was. It may have been caused by the same reason above, that the use of MAX_TILE_AREA_LEVEL_7_AND_ABOVE in the av1_get_tile_limits() function did not result in any difference.

You can apply the following patch to libaom 3.6.0 to see if a particular width x height is affected by this bug:

diff --git a/av1/common/tile_common.c b/av1/common/tile_common.c
index 45f4d0c..5897462 100644
--- a/av1/common/tile_common.c
+++ b/av1/common/tile_common.c
@@ -63,6 +63,17 @@ void av1_get_tile_limits(AV1_COMMON *const cm) {
   tiles->max_log2_rows = tile_log2(1, AOMMIN(sb_rows, MAX_TILE_ROWS));
   tiles->min_log2 = tile_log2(max_tile_area_sb, sb_cols * sb_rows);
   tiles->min_log2 = AOMMAX(tiles->min_log2, tiles->min_log2_cols);
+
+  if (use_level_7_above) {
+    const int max_tile_area_sb_old = MAX_TILE_AREA >> (2 * sb_size_log2);
+    int min_log2 = tile_log2(max_tile_area_sb_old, sb_cols * sb_rows);
+    min_log2 = AOMMAX(min_log2, tiles->min_log2_cols);
+    if (tiles->min_log2 != min_log2) {
+      fprintf(stderr,
+              "The use of MAX_TILE_AREA_LEVEL_7_AND_ABOVE results in a "
+              "difference\n");
+    }
+  }
 }

 void av1_calculate_tile_cols(const SequenceHeader *const seq_params,