[Task] - Screen JPEG decode using DMA

[Madour]

Description

Right now, image and video display is done using JPEG decoding in "polling" mode, which is very slow (~100ms for 1 image). The better way is to use JPEG decoding with DMA.

Subtasks

• [x] Display an image using HAL_JPEG_Decode_DMA
• [x] Play a video using HAL_JPEG_Decode_DMA
• [x] Integrate decode_dma.c functions in video driver
• [x] Optimize performance
  • [x] Double frame buffer
  • [x] #284
  • [x] Use DSI in adapted command mode -> frame buffer in GRAM -> faster data transfer = better fps

[Madour]

Current problem :

Program freeze for unknown reason in JPEG_DMA_StartProcess (private function from stm32f7xx_hal_jpeg.c) at macro instruction :

JPEG_ENABLE_DMA(hjpeg, JPEG_DMA_IDMA | JPEG_DMA_ODMA);

which expands to

hjpeg->Instance->CR |= ((JPEG_CR_IDMAEN | JPEG_CR_ODMAEN) & JPEG_DMA_MASK)

JPEG_CR_IDMAEN and JPEG_CR_ODMAEN are respectively "Input DMA enable" and "Output DMA enable". They are located at bit 11 and 12 of JPEG control register. Setting these bits to 1 should enable DMA process, but the program freeze (no crash, infinite freeze)

JPEG_DMA_MASK is 0x1800 and used to mask bits 11 and 12.

Expected behavior :

The 2 bits (IDMAEN and ODMAEN) of the JPEG control registers are set to 1, which will allow the DMA to start processing the JPEG data.

---

Call stack :

main
JPEG_Decode_DMA (in decode_dma.c)
HAL_JPEG_Decode_DMA (in stm32f7xx_hal_jpeg.c)
JPEG_DMA_StartProcess (in stm32f7xx_hal_jpeg.c)

[ladislas]

If it's a bug, it can be reported here: https://github.com/STMicroelectronics/STM32CubeF7

If not the ST forum is usually pretty good.

The important question is always: does it work as expected with just Cube and no mbed? When adding mbed when does it start to break?

The issue could come from the way CubeF7 is merged into mbed, some parts can go missing or some macros may not be defined correctly. It would not be the first time.

The ST team is very active on mbedOS repository so it's also a good idea to asks questions there or on mbed forum as well.

[YannLocatelli]

If I remember there are two functions called in JPEG Decode example of Cube but not in our example under mbed.

https://github.com/STMicroelectronics/STM32CubeF7/blob/c7c5ec99c7482ea8bcdbf0a869c930af4547088f/Projects/STM32F769I-Discovery/Examples/JPEG/JPEG_DecodingUsingFs_DMA/Src/main.c#L77 CPU_CACHE_Enable();

https://github.com/STMicroelectronics/STM32CubeF7/blob/c7c5ec99c7482ea8bcdbf0a869c930af4547088f/Projects/STM32F769I-Discovery/Examples/JPEG/JPEG_DecodingUsingFs_DMA/Src/main.c#L92 SystemClock_Config();

For the second one, it is "automatically" called by mbed, but might be a slightly different, maybe the clock of DMA2 is not started in mbed but it is on Cube.

[Madour]

In HAL_JPEG_MspInit the clocks are enabled :

// Enable JPEG clock   
__HAL_RCC_JPEG_CLK_ENABLE();

// Enable DMA clock
__HAL_RCC_DMA2_CLK_ENABLE();

This function also initializes the DMA streams 3 and 4 and associates them to the JPEG handle. I also put a log_info in this function to make sure it is called, and it is indeed.

I also found out that the problem comes from setting the IDMAEN bit :

hjpeg->Instance->CR |= JPEG_CR_ODMAEN; // works
hjpeg->Instance->CR |= JPEG_CR_IDMAEN; // freeze

I made a post on ST forums. and I am trying to double check everything in the meantime (https://community.st.com/s/question/0D53W00000sAVYQSA4/cannot-enable-jpeg-decode-with-dma-on-stm32f76xx).

---

For the cache, they are enabled in mbed_sdk_init just like in the ST example :

// This function is called after RAM initialization and before main.
void mbed_sdk_init()
{
#if defined(__ICACHE_PRESENT) /* STM32F7 */
    // The mbed_sdk_init can be called either during cold boot or during
    // application boot after bootloader has been executed.
    // In case the bootloader has already enabled the cache,
    // is is needed to not enable it again.
    if ((SCB->CCR & (uint32_t)SCB_CCR_IC_Msk) == 0) { // If ICache is disabled
        SCB_EnableICache();
    }
    if ((SCB->CCR & (uint32_t)SCB_CCR_DC_Msk) == 0) { // If DCache is disabled
        SCB_EnableDCache();
    }
#endif /* __ICACHE_PRESENT */

...

}

---

Investigations :

DMA registers are set correctly (peripheral address, control register ...). It means DMA was probably correctly initialized ... JPEG also correctly initialized, everything is as it should be.

---

This bug maybe comes from Mbed OS and the way it overrides configurations. Post on mbed forums : https://forums.mbed.com/t/stm32f769-jpeg-decode-with-dma-bug/13569

[Madour]

Image + Video display using JPEG Decoder in DMA mode works finally. Right now, I am using decode_dma.h/.c from ST. My last attempt to integrate it into LKCoreJPEG was not successfull, so i'll have to try again now that I'm sure its functionnal.

However, the performance is not very good yet, 100 to 110 ms per frame (130ms in polling mode), was expecting less than the half of this time. Will have to see why it is not fast like on the SMT32F769ni board.

One difference between the board and Leka is the SD card protocol/interface used, and SPI is slower than SDMMC usually (I think) ... f_read : 1 ms on STM32F7 board vs 5 ms on Leka_V1_2 ,

[ladislas]

see #456