marbalon commented 2 years ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.0-beta1-427-g4532e6e0b2

Operating System used.

Windows

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

No response

Development Kit.

Custom board

Power Supply used.

External 5V

What is the expected behavior?

Keep RGB LCD working/refreshed while SPI flash is used to store the data.

What is the actual behavior?

I have a project with ESP32S3 + LCD 800x480 over RGB + 8MByte PSRAM. ESP-IDF 5.0.

Just to test I've just stripped my project to display a static images on the screen using:

esp_lcd_panel_draw_bitmap(panel_handle, 0, 0, 800, 480, bg_buff);

And of course, it is displayed correctly. But when my application needs to do any SPI writes LCD is not refreshed correctly - it starts to move right if this is only one write, If I need to clear more data in the flash it is basically scrolling.

I know that the program is running also from SPI flash but I thought that if all functions related to LCD and DMA are marked as IRAM_ATTR, the LCD and DMA interrupts can be executed as they should.

So now there is no way to keep LCD working when flash has to be used for storing data or OTA upgrade. I know OTA is not used so often, but my application just downloads images time to time from the internet and display it in a loop but it is impossible to do this in a correct way.

If this is not possible we have an idea to use a second flash as storage, but the question is - can we use two octal SPI flash chips on the same BUS? Can this work as a solution?

Edit.

Or maybe there is an option to run LCD driver from RAM memory - but I set all possible flags in menuconfig related to IRAM option for GDMA, LCD and SPI driver and this did not change anything.

Steps to reproduce.

Debug Logs.

No response

More Information.

No response

marbalon commented 2 years ago

I've just found information in espressif doc that when SPI flash is writen external SPI flash is also not accessible if I understand this correctly:

Restrictions The use of external RAM has a few restrictions: When disabling flash cache (for example, because the flash is being written to), the external RAM also becomes inaccessible; any reads from or writes to it will lead to an illegal cache access exception. This is also the reason that ESP-IDF will never allocate a tasks stack in external RAM.

So what I did just for the test I disabled the double frame buffer and then allocate LV buffers for 50 lines in the internal memory. But this is also not a solution because internal frame buffer for RGB panel is for a full screen and function lcd_rgb_panel_alloc_frame_buffers() can't allocate memory for display 800x480 inside internal memory so this is the reason why this is also not helping.

Anyone have any other idea how to keep LCD refreshed when SPI flash is used? Maybe not the best solution but a solution is that I can display only static text while the device is updated, so I have everything program/flash memory.

Any suggestions? Do you think a solution with second memory like a second SPI flash or SD card for images is good option and writing to second flash memory is not blocking the execution of the main program?

GTCLive commented 2 years ago

Suggestions; try implementing semaphore/mutex logic and locking mechanisms, and review if this is a bug or an implementation issue.

marbalon commented 2 years ago

Suggestions; try implementing semaphore/mutex logic and locking mechanisms, and review if this is a bug or an implementation issue.

After reading the manual I think it is not a bug - it looks like it is not possible to use the main flash when LCD is used. I think mutex will also not help because the main program and DMA handlers have to fill LCD data all the time.

The main question stays the same - Does the second flash memory connected in parallel with the main SPI chip is a solution in this case?

suda-morris commented 2 years ago

@marbalon have you tried to enable the LCD_RGB_ISR_IRAM_SAFE option?

We have a test case for this, to ensure the RGB LCD driver won't crash the application when doing main flash operations like erase, write... But you have to enable that Kconfig option.

marbalon commented 2 years ago

@marbalon have you tried to enable the LCD_RGB_ISR_IRAM_SAFE option?

Yes, and also the same DMA, and SPI interrupt - nothing changed. But I will try to run the test you suggested.

suda-morris commented 2 years ago

@marbalon have you tried to enable the LCD_RGB_ISR_IRAM_SAFE option?

Yes, and also the same DMA, and SPI interrupt - nothing changed. But I will try to run the test you suggested.

More info, in the test, we enabled these Kconfigs: https://github.com/espressif/esp-idf/blob/master/components/esp_lcd/test_apps/rgb_lcd/sdkconfig.ci.iram_safe

marbalon commented 2 years ago

Ok, I've just done some tests, and it looked pretty optimistic at the beginning. A simple test passed but I don't see too much so I've added a for loop to see more. And LCD was refreshed perfectly. Then I decided to initialize spiffs file system and just write data in a loop. And it looks like when the spiffs have to flush some data to the spi flash LCD have a problem with refreshing.

Here is my modified code, and video below. You can see every few seconds the picture is shifted to the left, and also a bit worst effect from time to time (eg in 11sec).

https://photos.app.goo.gl/wtBRn5gfHZmK4MH39

Sorry for mess below with the code but github don't want to close everything in to one code.

`

define TEST_FILE "/spiffs/test.file"

TEST_CASE("lcd_rgb_panel_iram_safe", "[lcd]") { uint8_t *img = malloc(TEST_IMG_SIZE); TEST_ASSERT_NOT_NULL(img); uint32_t callback_calls = 0;

unlink(TEST_FILE);

FILE* f = fopen(TEST_FILE, "wb");
if (f == NULL) {
    printf("Failed to open file test file\n");
    return;
}

printf("initialize RGB panel with stream mode\r\n");
esp_lcd_panel_handle_t panel_handle = test_rgb_panel_initialization(16, 16, 0, false, test_rgb_panel_count_in_callback, &callback_calls);
printf("flush one clock block to the LCD\r\n");
printf("The LCD driver should keep flushing the color block in the background (as it's in stream mode)\r\n");
printf("the RGB ISR handle should keep working while the flash cache is disabled\r\n");

for (int i = 0; i < 2000; i++) {
    uint8_t color_byte = esp_random() & 0xFF;
    int x_start = esp_random() % (TEST_LCD_H_RES - 100);
    int y_start = esp_random() % (TEST_LCD_V_RES - 100);
    memset(img, color_byte, TEST_IMG_SIZE);
    esp_lcd_panel_draw_bitmap(panel_handle, x_start, y_start, x_start + 100, y_start + 100, img);

    // read/write the SPI Flash by NVS APIs, the LCD driver should stay work
    //printf("disable the cache for a while\r\n");
    //test_disable_flash_cache();
    fwrite(&callback_calls, sizeof(callback_calls), 1, f);
    vTaskDelay(pdMS_TO_TICKS(20));
    printf("callback calls: %"PRIu32"\r\n", callback_calls);
}

printf("delete RGB panel\r\n");
TEST_ESP_OK(esp_lcd_panel_del(panel_handle));
free(img);
fclose(f);

} `

suda-morris commented 2 years ago

I'm afraid this is expected because when you dong file operations to the main flash (vis SPI1 bus), EDMA can't fetch the framebuffer from PSRAM (on SPI0 bus). The SPI0 and SPI1 actually share the same hardware, exclusively.

For now, there're two ways to save the shifted screen:

LCD_RGB_RESTART_IN_VSYNC Kconfig will restart the DMA, so after your flash operation done, you screen should restore to normal automatically, but with an extra overload because of the "restart" operation.
esp_lcd_rgb_panel_restart API, you can call this function whenever you think the screen is shifted.

marbalon commented 2 years ago

@suda-morris, thanks for your suggestions. So sadly you agree with me that it is impossible to keep LCD refreshed when SPI flash is used.

So If I connect the second flash to the same SPI and write to this flash instead of the main flash it will also not help? But if I use SPI2/3 bus with different pins?

Another option is an SD card - do you think this should also work?

We have to check if we have enough pins to use any of these peripherals.

marbalon commented 2 years ago

Ok, answering my self - idea with second flash connected to SPI3 using QIO is working fine. No problem with LCD update while writing to external flash, except new bug reported here https://github.com/espressif/esp-idf/issues/10060.

So we can close this issue but I think some extra information should be added to the documentation about this limit.

KaeLL commented 2 years ago

The SPI0 and SPI1 actually share the same hardware

Then what exactly isn't shared that's enough to differentiate them @suda-morris?

suda-morris commented 2 years ago

FYI, this diagram is for ESP32S2, but should be similar to ESP32S3. At one time, there could be only one consumer of the BUS.

espressif / esp-idf

LCD over RGB interface is not refreshed while writing to SPI flash (IDFGH-8561) #10010

Answers checklist.

IDF version.

Operating System used.

How did you build your project?

If you are using Windows, please specify command line type.

Development Kit.

Power Supply used.

What is the expected behavior?

What is the actual behavior?

Steps to reproduce.

Debug Logs.

More Information.

define TEST_FILE "/spiffs/test.file"