Decoding JPEG with SIMD

[modi12jin]

@cgreening JPEG decoding with SIMD, currently the whole frame, cannot be partial, there will be more in the future

The only thing that needs attention is that the buffer must be 16-byte aligned. I tested 320240 with a box and it took an average of 42 ms to decode RGB565. The performance on Arduino is really not good. I remember that decoding 800480 under IDF took less than 50ms.

JPEGDEC seems to be 68ms

https://github.com/esp-arduino-libs/ESP32_JPEG/blob/master/examples/DecodeTest/DecodeTest.ino

[cgreening]

I'll take a look.

[cgreening]

It's very fast - but unfortunately it doesn't seem to work with all image sizes.

It works ok with the 320x240 image and a 160x120 image. But doesn't seem to work with some other size. Any idea what could be going on?

[cgreening]

It looks like the width has to be a multiple of 16 for some reason.

[cgreening]

Some pretty interesting results.

Library	Decode+Draw Time	Decode Time
Bodmer/JPEGDecoder	117ms	109ms
bitbank2/JPEGDEC	38ms	32ms
ESP32_JPEG	33ms	17ms

What can we conclude....

The ESP32 based decoding is really fast. But, the limiting factor is the DMA transfer to draw the image. Which is why we get a similar time for decode+draw to the JPEGDEC library.

The JPEGDEC library interleaves decoding and drawing, so while it's drawing one section of the image using DMA it's decoding the next section.

The ESP32_JPEG would use about half the CPU time so it's still quite interesting.

My only problem is the need to have the width of the image a multiple of 16 which doesn't seem to match what the header says is possible.

[modi12jin]

@cgreening You are right, 16-byte alignment is indeed required. Only the whole frame, not parts.

Partial decoding will be available in the near future. This is the information I got from the official

If you want to use it in IDF, you can refer to this link https://github.com/espressif/esp-dev-kits/tree/master/esp32-s3-lcd-ev-board%2Fexamples%2Fusb_camera_lcd%2Fcomponents%2Fesp_jpeg

Chen Liang test video https://x.com/moononournation/status/1705953255979888696?s=20

https://github.com/moononournation/ESP32-S3-BOX-3/tree/main/ImgViewerMjpeg_SIMD

The perfect combination of SIMD full screen decoding + RGB screen

Questions I asked before JPEGDEC https://github.com/bitbank2/JPEGDEC/issues/56

As an aside, SIMD can be used to speed up some LVGL drawing processes. https://github.com/espressif/esp-bsp/issues/154

[modi12jin]

@cgreening Thanks for the explainer video! I got the news that ESP32_JPEG will add relevant usage instructions, such as 16-byte alignment issues

[modi12jin]

@cgreening The JPEGDEC author has an update on this issue https://bitbanksoftware.blogspot.com/2024/01/surprise-esp32-s3-has-few-simd.html?m=1

[modi12jin]

@cgreening This is jpeg SIMD decoding, which is now partially supported. Sir, you can try it and see how it works

7_20231124_jpeg_block_decoder_esp32s3.zip