search

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.74k stars 7.3k forks source link

ESP32S3 LCD 和 Camera同时使用出现冲突 (IDFGH-11787) #12882

Open liruya opened 10 months ago

liruya commented 10 months ago

Answers checklist.

General issue report

esp-idf版本5.1.2. ESP32S3 LCD用i80接口或者RGB接口, 同时使用Camera, 就会出现这种情况. 如果先初始化LCD, 则Camera初始化失败, 如果先初始化Camera, 则LCD初始化失败. 如果LCD用SPI接口, 则没有问题. 定位到 esp_intr_alloc_intrstatus -> get_available_int 这里, 中断配置失败. LCD和Camera的中断配置出现冲突, 不知道怎么解决.

suda-morris commented 10 months ago

@liruya which esp32_camera version are you using? Can you share your initialization code for the LCD and the Camera?

I test the esp-idf/5.1.2 + esp32_camera v2.0.6 and didn't see this error.

liruya commented 10 months ago

@liruya which esp32_camera version are you using? Can you share your initialization code for the LCD and the Camera?

I test the esp-idf/5.1.2 + esp32_camera v2.0.6 and didn't see this error.

esp32-camera版本2.0.6, 单独使用Camera没有问题, 问题出在Camera和LCD i80接口/RGB接口同时使用时. Camera和LCD同时使用, 后初始化的会失败. 目前定位问题出在中断配置, 推测是因为LCD和Camera用的同一个中断源, 但是分别配置中断, 导致后初始化的会分配中断失败.

liruya commented 10 months ago

@liruya which esp32_camera version are you using? Can you share your initialization code for the LCD and the Camera?

I test the esp-idf/5.1.2 + esp32_camera v2.0.6 and didn't see this error.

void lcd_disp_init(void) {
#if LCD_PIN_BL >= 0
    ESP_LOGI(TAG, "Turn off LCD backlight");
    gpio_config_t bl_gpio_config = {
        .mode = GPIO_MODE_OUTPUT,
        .pin_bit_mask = BIT64(LCD_PIN_BL)
    };
    ESP_ERROR_CHECK(gpio_config(&bl_gpio_config));
    gpio_set_level(LCD_PIN_BL, LCD_BL_LEVEL_OFF);
#endif

    ESP_LOGI(TAG, "Initialize Intel 8080 bus");
    esp_lcd_i80_bus_handle_t i80_bus = NULL;
    esp_lcd_i80_bus_config_t bus_config = {
        .clk_src = LCD_CLK_SRC_PLL240M,
        .dc_gpio_num = LCD_PIN_DC,
        .wr_gpio_num = LCD_PIN_WR,
        .data_gpio_nums = {
            LCD_PIN_DATA0,
            LCD_PIN_DATA1,
            LCD_PIN_DATA2,
            LCD_PIN_DATA3,
            LCD_PIN_DATA4,
            LCD_PIN_DATA5,
            LCD_PIN_DATA6,
            LCD_PIN_DATA7,
            LCD_PIN_DATA8,
            LCD_PIN_DATA9,
            LCD_PIN_DATA10,
            LCD_PIN_DATA11,
            LCD_PIN_DATA12,
            LCD_PIN_DATA13,
            LCD_PIN_DATA14,
            LCD_PIN_DATA15
        },
        .bus_width = LCD_I80BUS_WIDTH,
        .max_transfer_bytes = LCD_WIDTH * 100 * sizeof(uint16_t),
        .psram_trans_align = 64,
        .sram_trans_align = 4
    };
    ESP_ERROR_CHECK(esp_lcd_new_i80_bus(&bus_config, &i80_bus));

    ESP_LOGI(TAG, "Install panel IO");
    esp_lcd_panel_io_handle_t io_handle = NULL;
    esp_lcd_panel_io_i80_config_t io_config = {
        .cs_gpio_num        = LCD_PIN_CS,
        .pclk_hz            = LCD_PCLK_HZ,
        .trans_queue_depth  = 10,
        .dc_levels          = {
            .dc_idle_level  = 0,
            .dc_cmd_level   = 0,
            .dc_dummy_level = 0,
            .dc_data_level  = 1
        },
        .flags              = {
            .swap_color_bytes = 0,      //  !LV_COLOR_16_SWAP
        },
        .on_color_trans_done    = lvgl_flush_ready,
        .user_ctx           = &lv_disp_drv,
        .lcd_cmd_bits       = LCD_CMD_BITS,
        .lcd_param_bits     = LCD_PARAM_BITS
    };
    ESP_ERROR_CHECK(esp_lcd_new_panel_io_i80(i80_bus, &io_config, &io_handle));

    ESP_LOGI(TAG, "Install LCD driver of st7789");
    esp_lcd_panel_dev_config_t panel_config = {
        .reset_gpio_num     = LCD_PIN_RST,
        .rgb_ele_order      = LCD_RGB_ELEMENT_ORDER_RGB,
        .bits_per_pixel     = 16
    };
    ESP_ERROR_CHECK(esp_lcd_new_panel_st7789(io_handle, &panel_config, &lcd_panel_handle));

    esp_lcd_panel_reset(lcd_panel_handle);
    esp_lcd_panel_init(lcd_panel_handle);

#if LCD_SWAP_XY
    esp_lcd_panel_swap_xy(lcd_panel_handle, true);
#endif
#if LCD_MIRROR_X || LCD_MIRROR_Y
    esp_lcd_panel_mirror(lcd_panel_handle, LCD_MIRROR_X, LCD_MIRROR_Y);
#endif
    esp_lcd_panel_set_gap(lcd_panel_handle, 0, 0);

    // user can flush pre-defined pattern to the screen before we turn on the screen or backlight
    ESP_ERROR_CHECK(esp_lcd_panel_disp_on_off(lcd_panel_handle, true));
}

static camera_config_t camera_config = {
    .pin_pwdn       = CAM_PIN_PWDN,
    .pin_reset      = CAM_PIN_RST,
    .pin_xclk       = CAM_PIN_XCLK,
    .pin_sccb_sda   = CAM_PIN_SDA,
    .pin_sccb_scl   = CAM_PIN_SCL,

    .pin_pclk       = CAM_PIN_PCLK,
    .pin_href       = CAM_PIN_HSYNC,
    .pin_vsync      = CAM_PIN_VSYNC,
    .pin_d7         = CAM_PIN_D7,
    .pin_d6         = CAM_PIN_D6,
    .pin_d5         = CAM_PIN_D5,
    .pin_d4         = CAM_PIN_D4,
    .pin_d3         = CAM_PIN_D3,
    .pin_d2         = CAM_PIN_D2,
    .pin_d1         = CAM_PIN_D1,
    .pin_d0         = CAM_PIN_D0,

    .xclk_freq_hz   = 2000000,
    .ledc_timer     = LEDC_TIMER_0,
    .ledc_channel   = LEDC_CHANNEL_0,

    .pixel_format   = PIXFORMAT_RGB565,
    .frame_size     = FRAMESIZE_QVGA,

    .jpeg_quality   = 12,       //  0-63, for OV series camera sensors, lower number means higher quality
    .fb_count       = 1,        //When jpeg mode is used, if fb_count more than one, the driver will work in continuous mode.
    .fb_location    = CAMERA_FB_IN_PSRAM,
    .grab_mode      = CAMERA_GRAB_WHEN_EMPTY,
    .sccb_i2c_port  = 0
};

static esp_err_t camera_init(void) {
#if     CAM_PIN_PWDN >= 0
    gpio_config_t pwdn_conf = {
        .mode   = GPIO_MODE_OUTPUT,
        .pin_bit_mask   = BIT64(CAM_PIN_PWDN)
    };
    gpio_config(&pwdn_conf);
    gpio_set_level(CAM_PIN_PWDN, 0);
#endif

    esp_err_t ret = esp_camera_init(&camera_config);
    if (ret != ESP_OK) {
        ESP_LOGE(TAG, "camera init failed!");
        return ret;
    }

    return ESP_OK;
}
suda-morris commented 10 months ago 
/*
 * SPDX-FileCopyrightText: 2010-2022 Espressif Systems (Shanghai) CO LTD
 *
 * SPDX-License-Identifier: CC0-1.0
 */

#include <stdio.h>
#include <inttypes.h>
#include "sdkconfig.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_camera.h"
#include "esp_lcd_panel_io.h"

#define TAG "example"

void lcd_disp_init(void)
{
    ESP_LOGI(TAG, "Initialize Intel 8080 bus");
    esp_lcd_i80_bus_handle_t i80_bus = NULL;
    esp_lcd_i80_bus_config_t bus_config = {
        .clk_src = LCD_CLK_SRC_PLL240M,
        .dc_gpio_num = 0,
        .wr_gpio_num = 0,
        .data_gpio_nums = {
            0, 0, 0, 0, 0, 0, 0, 0
        },
        .bus_width = 8,
        .max_transfer_bytes = 200 * 100 * sizeof(uint16_t),
        .psram_trans_align = 64,
        .sram_trans_align = 4
    };
    ESP_ERROR_CHECK(esp_lcd_new_i80_bus(&bus_config, &i80_bus));

    ESP_LOGI(TAG, "Install panel IO");
    esp_lcd_panel_io_handle_t io_handle = NULL;
    esp_lcd_panel_io_i80_config_t io_config = {
        .cs_gpio_num        = 0,
        .pclk_hz            = 10000000,
        .trans_queue_depth  = 10,
        .dc_levels          = {
            .dc_idle_level  = 0,
            .dc_cmd_level   = 0,
            .dc_dummy_level = 0,
            .dc_data_level  = 1
        },
        .lcd_cmd_bits       = 8,
        .lcd_param_bits     = 8
    };
    ESP_ERROR_CHECK(esp_lcd_new_panel_io_i80(i80_bus, &io_config, &io_handle));
}

static camera_config_t camera_config = {
    .pin_pwdn       = 1,
    .pin_reset      = 1,
    .pin_xclk       = 1,
    .pin_sccb_sda   = 1,
    .pin_sccb_scl   = 2,

    .pin_pclk       = 1,
    .pin_href       = 1,
    .pin_vsync      = 1,
    .pin_d7         = 1,
    .pin_d6         = 1,
    .pin_d5         = 1,
    .pin_d4         = 1,
    .pin_d3         = 1,
    .pin_d2         = 1,
    .pin_d1         = 1,
    .pin_d0         = 1,

    .xclk_freq_hz   = 2000000,
    .ledc_timer     = 0,
    .ledc_channel   = 0,

    .pixel_format   = PIXFORMAT_RGB565,
    .frame_size     = FRAMESIZE_QVGA,

    .jpeg_quality   = 12,       //  0-63, for OV series camera sensors, lower number means higher quality
    .fb_count       = 1,        //When jpeg mode is used, if fb_count more than one, the driver will work in continuous mode.
    .grab_mode      = CAMERA_GRAB_WHEN_EMPTY,
};

void app_main(void)
{
    lcd_disp_init();
    ESP_ERROR_CHECK(esp_camera_init(&camera_config));
}

yes, I still can't see the issue with the above code. Please ignore the GPIO settings. I just want to test the esp_lcd_new_i80_bus and esp_camera_init can work at the same time because both of them will call esp_intr_alloc_intrstatus

liruya commented 10 months ago

LCD和Camera初始化冲突找到解决办法, 确实是配置中断的冲突, LCD和Camera中断用同一个中断源, 导致分配中断失败. 解决办法是LCD中断和Camera中断分配到不同的CPU. 比如LCD在CPU0 任务里初始化, Camera在CPU1任务里初始化. 另外发现必须先初始化Camera再初始化LCD, 否则还是有异常, 这个没发现什么原因.

目前LCD用80接口基本正常, 可以初始化成功, 摄像头实时采集并显示到LCD. 但是LCD用RGB接口, 可以初始化成功, 但是Camera采集失败, 提示Failed to get the frame on time!

suda-morris commented 9 months ago

Hi @liruya Thanks for keeping debug! The camera will allocate the interrupt with ESP_INTR_FLAG_IRAM. But this is not the case for the RGB LCD driver. In the RGB LCD driver, we don't use that flag unless the Kconfig LCD_RGB_ISR_IRAM_SAFE is enabled. That means, if you enabled that flag, then you can initialize the RGB LCD and camera driver from the same CPU core.

But I doubt if the camera should use ESP_INTR_FLAG_IRAM in the first place. I will ask the camera author to evaluate this.

AxelLin commented 9 months ago

@suda-morris

If my understanding is correct: gpio_install_isr_service() only takes effect for the first call. (The second call will return ESP_ERR_INVALID_STATE, "GPIO isr service already installed")

I'm wondering what happened if 2 components that both have called gpio_install_isr_service() with different intr_alloc_flags. Isn't this racy?

For example, if I have code using examples/bluetooth/esp_ble_mesh/common_components/button/button.c which calls gpio_install_isr_service(0); I also use esp32-camera driver which calls gpio_install_isr_service(ESP_INTR_FLAG_LOWMED | ESP_INTR_FLAG_IRAM);

Then different call sequence has different intr_alloc_flags setting. And if any of the component calls gpio_uninstall_isr_service(), both components stop working?

CC @me-no-dev for the question in https://github.com/espressif/esp-idf/issues/12882#issuecomment-1899873402

WangYuxin-esp commented 9 months ago

There are two areas worth investigating:

me-no-dev commented 9 months ago

@WangYuxin-esp thanks for the pointers!

@AxelLin please try https://github.com/espressif/esp32-camera/pull/629

me-no-dev commented 9 months ago

@suda-morris the flag is there to help speed things up a bit. ESP32 needs it to convert I2S samples to actual data. I can make it optional for S2 and S3 with LCD_CAM_ISR_IRAM_SAFE.

@AxelLin OK if I add it to the same PR for you to try?

AxelLin commented 7 months ago

@liruya Did you try esp32-camera v2.0.8? Is this still an issue?