Still image encoding is expensive

[kornelski]

Describe the bug Encoding of AV1 appears to be much more CPU-intensive than of other image formats at comparable compression ratios.

Using kodim08 as an example:

At speed 4 and below, rav1e beats WebP in file size and quality, but is an order of magnitude slower to encode. At speed 6, for this image, it's still much slower than cwebp, but generates a larger file with lower quality. At speed 10, the quality/filesize is comparable with MozJPEG, but twice as slow. The results vary depending on image and quality measurement algorithm, but the trend holds in general.

I've also compared to AOM using libavif example, and its encoding speed/quality is in the same ballpark as rav1e.

To Reproduce

cargo install cavif # 1.1.1
cargo install dssim --features=avif,webp # 3.1.0
time cavif --speed 1 --quality 57 -f kodim08.png -o kodim08.s1.avif
time cavif --speed 4 --quality 57 -f kodim08.png -o kodim08.s4.avif
time cavif --speed 6 --quality 57.4 -f kodim08.png -o kodim08.s6.avif
time cavif --speed 10 --quality 58.4 -f kodim08.png -o kodim08.s10.avif
time cwebp kodim08.png -o kodim08.webp
dssim kodim08.png kodim08.s*.avif kodim08.webp
ls -lS kodim08.*

Expected behavior I hoped the encoding would be faster. Have I used the encoder incorrectly? Am I missing some build setting? Is there a room for improvement in rav1e, or is this encoding cost typical for AV1?

Required Information Version: 0.5.0-alpha

[kornelski]

I've looked at existing detailed settings. I've been testing still images at quality comparable to JPEG q=80 (qualities normalized with DSSIM), so my conclusions may not apply to videos and low quality images.

My observations:

• rdo_tx_decision murders speed. It is somewhat beneficial for compression, but at 3x cost of encoding it should probably be reserved for speed=1 only.

• partition_range makes a significant difference for speed and quality. There's probably a lot to be gained from a smarter block size selection.

  • 8-32 and 4-16 are overall the best for quality, and give a decent speed.
  • 4-64 rarely helps, and often gives worse compression than 4-16.
  • 16-16 is a 2x improvement in speed, but hurts compression. It's still (barely) above WebP's, so they may be a good option for the fastest compression setting.
  • 32-32 is about same speed as 16-16, but much worse for quality. Bad idea to use it.
  • 64-64 is awful and should never be used. Generally, I'm not seeing benefits of big blocks.

• CDEF improves compression by a few %, but costs a few % in encoding speed too. Probably OK to disable at high speeds.

• LRF sometimes improves compression, sometimes hurts. Probably OK to always disable when encoding with high quality.

• encode_bottomup = true sometimes helps compression, but it highly depends on image content. It can backfire too.

• sgr_complexity = Full is not beneficial (worse quality, worse speed)

• non_square_partition and fast_deblock don't seem to have any effect.

• prediction_modes = Complex should probably be reserved for slower speeds 1-3 only. I presume ComplexKeyframes is applied to still images, and it's too slow for speed==6.