Use LVGL Direct mode in ESP32-S3 RGB panel example (IDFGH-7561)

[itavero]

Is your feature request related to a problem? Please describe.

When using the RGB LCD panel peripheral, I still see quite some "tearing" due to parts of the buffer being updated. I think this is partly due to how LVGL and the RGB peripheral have been integrated.

Describe the solution you'd like

The integration between LVGL and the RGB peripheral should be improved. As described in my forum post, for these kind of peripherals it makes sense to use one screen-sized buffer and use Direct mode in LVGL. This will make sure that you only get a trigger once all the updates have been rendered.

Another thing that should probably be added is some synchronization/alignment to make sure that the actual "copying" of the buffer is done after the frame has been "transferred" to the RGB panel and before a new frame transfer is started. In my forum post, I ask if I can use on_frame_trans_done for this. It has been "removed" from the example in the master branch, I believe due to an issue with the hardware, but I'm not 100% sure on that.

Note that the LVGL buffer probably also needs to be placed in the external PSRAM, as it will typically be too large for the internal RAM.

Describe alternatives you've considered

I tried using it in the same "partial buffer" mode as the example, but for us the resulting experience when there are animations all over the screen are not acceptable. Too much visible tearing/glitches occur.

I've not really tried any other examples, but I'm open to suggestions to get the experience as smooth as possible.

Additional context

• I'm using this panel and dev kit for my tests, which I got from AliExpress (currently the only dev kit I could find that actually uses the RGB peripheral of the ESP32-S3.
• My post on the Espressif forum.
• An older forum post on the LVGL forum about how tearing can be prevented.
  • I interpret it as: when the flush callback is called from LVGL, wait until the current frame is done before updating the peripherals frame buffer and calling lv_disp_flush_ready
  • Perhaps render_start_cb can also be used to do some optimizations.

[tore-espressif]

@itavero Thanks for the detailed bug report and letting us know about this.

Unfortunately, there can be various reason for glitches with RGB interface. Having one screen-sized buffer in PSRAM a direct mode set in LVGL can definitely help. If your board allows it, set PSRAM frequency to 80MHz and octal mode to get maximum throughput.

The glitches usually depend on the RGB panel itself. Then, this section of configuration code is critical https://github.com/espressif/esp-idf/blob/master/examples/peripherals/lcd/rgb_panel/main/rgb_lcd_example_main.c#L116-128 For my particular LCD panel, I have to set .flags.hsync_idle_low = true, to get uncorrupted image. Try experimenting with theses settings and/or check you LCD panel datasheet.

Could you share some images/short videos of the glitches?

We also implemented some optimizations recently, that will be made public on GitHub within few day. I'll ping you when it is ready

[itavero]

Short update: I got some feedback from @kmatch98 on the ESP32.com forum post. Based on that I might a minor change to both the ESP-IDF and LVGL, so that LVGL is in control of when the RGB peripheral is transferring the frame. That way I can prevent LVGL from writing to the frame buffer while the RGB peripheral is busy with transferring it. This resolved the "tearing" I saw before. Besides that I also switched back to using two buffers about 10% of the entire frame size in internal RAM and not using Direct mode in LVGL anymore.

I'm not sure if this will be the solution, but if it is, then it might be nice to have a new "official" API in the ESP-IDF that basically exposes the lcd_rgb_panel_start_transmission method that is currently static/internal and a way to prevent ESP-IDF from calling that function itself.

The change I have in the ESP-IDF locally right now is:

diff --git a/components/esp_lcd/include/esp_lcd_panel_rgb.h b/components/esp_lcd/include/esp_lcd_panel_rgb.h
index 73d8c0d65f..1281d6584c 100644
--- a/components/esp_lcd/include/esp_lcd_panel_rgb.h
+++ b/components/esp_lcd/include/esp_lcd_panel_rgb.h
@@ -122,6 +122,13 @@ typedef struct {
  */
 esp_err_t esp_lcd_new_rgb_panel(const esp_lcd_rgb_panel_config_t *rgb_panel_config, esp_lcd_panel_handle_t *ret_panel);

+/**
+ * @brief Start transmission of RGB data to LCD panel.
+ * @attention Added for testing purpose.
+ * @param panel  LCD panel handle
+ */
+void esp_lcd_rgb_panel_start_transmission(esp_lcd_panel_handle_t panel);
+
 #endif // SOC_LCD_RGB_SUPPORTED

 #ifdef __cplusplus
diff --git a/components/esp_lcd/src/esp_lcd_rgb_panel.c b/components/esp_lcd/src/esp_lcd_rgb_panel.c
index 40fc6778a0..daeafd5adf 100644
--- a/components/esp_lcd/src/esp_lcd_rgb_panel.c
+++ b/components/esp_lcd/src/esp_lcd_rgb_panel.c
@@ -225,6 +225,15 @@ err:
     return ret;
 }

+void esp_lcd_rgb_panel_start_transmission(esp_lcd_panel_handle_t panel) {
+    esp_rgb_panel_t *rgb_panel = __containerof(panel, esp_rgb_panel_t, base);
+
+    // restart the new transmission
+    if (!rgb_panel->flags.stream_mode) {
+        lcd_rgb_panel_start_transmission(rgb_panel);
+    }
+}
+
 static esp_err_t rgb_panel_del(esp_lcd_panel_t *panel)
 {
     esp_rgb_panel_t *rgb_panel = __containerof(panel, esp_rgb_panel_t, base);
@@ -338,7 +347,8 @@ static esp_err_t rgb_panel_draw_bitmap(esp_lcd_panel_t *panel, int x_start, int

     // restart the new transmission
     if (!rgb_panel->flags.stream_mode) {
-        lcd_rgb_panel_start_transmission(rgb_panel);
+      // NOTE: Not do this for now.. handled by application
+      //   lcd_rgb_panel_start_transmission(rgb_panel);
     }

     return ESP_OK;

If your board allows it, set PSRAM frequency to 80MHz and octal mode to get maximum throughput.

I believe it has SiP PSRAM, so I'll double check the configured frequency. Increasing this would also increase the maximum pixel clock frequency, right? The example in ESP-IDF currently mentions 12 MHz I think, on which PSRAM frequency is this based?

Could you share some images/short videos of the glitches?

Unfortunately these are hard to capture with the cameras I have on hand.

We also implemented some optimizations recently, that will be made public on GitHub within few day. I'll ping you when it is ready

Looking forward to that!

[tore-espressif]

Thank you for the provided code, it's good to hear that it works for you. I'll double check whether we can use the on_frame_trans_done

Looking forward to that!

The fix is available on our master branch now!

[itavero]

The fix is available on our master branch now!

Just tried it out, together with my aforementioned changes to "sync" access to the frame buffer, and the performance is indeed way better. On one of our demos we got about 11 to 12 FPS before and now it's about 24 FPS (which looks way better). Thanks for the update.

[weilian1977]

关注一下