search

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.37k stars 7.21k forks source link

ESP32S3 LCD 和 Camera同时使用出现冲突 (IDFGH-11787) #12882

Open liruya opened 8 months ago

liruya commented 8 months ago

Answers checklist.

General issue report

esp-idf版本5.1.2. ESP32S3 LCD用i80接口或者RGB接口, 同时使用Camera, 就会出现这种情况. 如果先初始化LCD, 则Camera初始化失败, 如果先初始化Camera, 则LCD初始化失败. 如果LCD用SPI接口, 则没有问题. 定位到 esp_intr_alloc_intrstatus -> get_available_int 这里, 中断配置失败. LCD和Camera的中断配置出现冲突, 不知道怎么解决.

suda-morris commented 8 months ago

@liruya which esp32_camera version are you using? Can you share your initialization code for the LCD and the Camera?

I test the esp-idf/5.1.2 + esp32_camera v2.0.6 and didn't see this error.

liruya commented 8 months ago

@liruya which esp32_camera version are you using? Can you share your initialization code for the LCD and the Camera?

I test the esp-idf/5.1.2 + esp32_camera v2.0.6 and didn't see this error.

esp32-camera版本2.0.6, 单独使用Camera没有问题, 问题出在Camera和LCD i80接口/RGB接口同时使用时. Camera和LCD同时使用, 后初始化的会失败. 目前定位问题出在中断配置, 推测是因为LCD和Camera用的同一个中断源, 但是分别配置中断, 导致后初始化的会分配中断失败.

liruya commented 8 months ago

@liruya which esp32_camera version are you using? Can you share your initialization code for the LCD and the Camera?

I test the esp-idf/5.1.2 + esp32_camera v2.0.6 and didn't see this error.

void lcd_disp_init(void) {
#if LCD_PIN_BL >= 0
    ESP_LOGI(TAG, "Turn off LCD backlight");
    gpio_config_t bl_gpio_config = {
        .mode = GPIO_MODE_OUTPUT,
        .pin_bit_mask = BIT64(LCD_PIN_BL)
    };
    ESP_ERROR_CHECK(gpio_config(&bl_gpio_config));
    gpio_set_level(LCD_PIN_BL, LCD_BL_LEVEL_OFF);
#endif

    ESP_LOGI(TAG, "Initialize Intel 8080 bus");
    esp_lcd_i80_bus_handle_t i80_bus = NULL;
    esp_lcd_i80_bus_config_t bus_config = {
        .clk_src = LCD_CLK_SRC_PLL240M,
        .dc_gpio_num = LCD_PIN_DC,
        .wr_gpio_num = LCD_PIN_WR,
        .data_gpio_nums = {
            LCD_PIN_DATA0,
            LCD_PIN_DATA1,
            LCD_PIN_DATA2,
            LCD_PIN_DATA3,
            LCD_PIN_DATA4,
            LCD_PIN_DATA5,
            LCD_PIN_DATA6,
            LCD_PIN_DATA7,
            LCD_PIN_DATA8,
            LCD_PIN_DATA9,
            LCD_PIN_DATA10,
            LCD_PIN_DATA11,
            LCD_PIN_DATA12,
            LCD_PIN_DATA13,
            LCD_PIN_DATA14,
            LCD_PIN_DATA15
        },
        .bus_width = LCD_I80BUS_WIDTH,
        .max_transfer_bytes = LCD_WIDTH * 100 * sizeof(uint16_t),
        .psram_trans_align = 64,
        .sram_trans_align = 4
    };
    ESP_ERROR_CHECK(esp_lcd_new_i80_bus(&bus_config, &i80_bus));

    ESP_LOGI(TAG, "Install panel IO");
    esp_lcd_panel_io_handle_t io_handle = NULL;
    esp_lcd_panel_io_i80_config_t io_config = {
        .cs_gpio_num        = LCD_PIN_CS,
        .pclk_hz            = LCD_PCLK_HZ,
        .trans_queue_depth  = 10,
        .dc_levels          = {
            .dc_idle_level  = 0,
            .dc_cmd_level   = 0,
            .dc_dummy_level = 0,
            .dc_data_level  = 1
        },
        .flags              = {
            .swap_color_bytes = 0,      //  !LV_COLOR_16_SWAP
        },
        .on_color_trans_done    = lvgl_flush_ready,
        .user_ctx           = &lv_disp_drv,
        .lcd_cmd_bits       = LCD_CMD_BITS,
        .lcd_param_bits     = LCD_PARAM_BITS
    };
    ESP_ERROR_CHECK(esp_lcd_new_panel_io_i80(i80_bus, &io_config, &io_handle));

    ESP_LOGI(TAG, "Install LCD driver of st7789");
    esp_lcd_panel_dev_config_t panel_config = {
        .reset_gpio_num     = LCD_PIN_RST,
        .rgb_ele_order      = LCD_RGB_ELEMENT_ORDER_RGB,
        .bits_per_pixel     = 16
    };
    ESP_ERROR_CHECK(esp_lcd_new_panel_st7789(io_handle, &panel_config, &lcd_panel_handle));

    esp_lcd_panel_reset(lcd_panel_handle);
    esp_lcd_panel_init(lcd_panel_handle);

#if LCD_SWAP_XY
    esp_lcd_panel_swap_xy(lcd_panel_handle, true);
#endif
#if LCD_MIRROR_X || LCD_MIRROR_Y
    esp_lcd_panel_mirror(lcd_panel_handle, LCD_MIRROR_X, LCD_MIRROR_Y);
#endif
    esp_lcd_panel_set_gap(lcd_panel_handle, 0, 0);

    // user can flush pre-defined pattern to the screen before we turn on the screen or backlight
    ESP_ERROR_CHECK(esp_lcd_panel_disp_on_off(lcd_panel_handle, true));
}

static camera_config_t camera_config = {
    .pin_pwdn       = CAM_PIN_PWDN,
    .pin_reset      = CAM_PIN_RST,
    .pin_xclk       = CAM_PIN_XCLK,
    .pin_sccb_sda   = CAM_PIN_SDA,
    .pin_sccb_scl   = CAM_PIN_SCL,

    .pin_pclk       = CAM_PIN_PCLK,
    .pin_href       = CAM_PIN_HSYNC,
    .pin_vsync      = CAM_PIN_VSYNC,
    .pin_d7         = CAM_PIN_D7,
    .pin_d6         = CAM_PIN_D6,
    .pin_d5         = CAM_PIN_D5,
    .pin_d4         = CAM_PIN_D4,
    .pin_d3         = CAM_PIN_D3,
    .pin_d2         = CAM_PIN_D2,
    .pin_d1         = CAM_PIN_D1,
    .pin_d0         = CAM_PIN_D0,

    .xclk_freq_hz   = 2000000,
    .ledc_timer     = LEDC_TIMER_0,
    .ledc_channel   = LEDC_CHANNEL_0,

    .pixel_format   = PIXFORMAT_RGB565,
    .frame_size     = FRAMESIZE_QVGA,

    .jpeg_quality   = 12,       //  0-63, for OV series camera sensors, lower number means higher quality
    .fb_count       = 1,        //When jpeg mode is used, if fb_count more than one, the driver will work in continuous mode.
    .fb_location    = CAMERA_FB_IN_PSRAM,
    .grab_mode      = CAMERA_GRAB_WHEN_EMPTY,
    .sccb_i2c_port  = 0
};

static esp_err_t camera_init(void) {
#if     CAM_PIN_PWDN >= 0
    gpio_config_t pwdn_conf = {
        .mode   = GPIO_MODE_OUTPUT,
        .pin_bit_mask   = BIT64(CAM_PIN_PWDN)
    };
    gpio_config(&pwdn_conf);
    gpio_set_level(CAM_PIN_PWDN, 0);
#endif

    esp_err_t ret = esp_camera_init(&camera_config);
    if (ret != ESP_OK) {
        ESP_LOGE(TAG, "camera init failed!");
        return ret;
    }

    return ESP_OK;
}
suda-morris commented 8 months ago 
/*
 * SPDX-FileCopyrightText: 2010-2022 Espressif Systems (Shanghai) CO LTD
 *
 * SPDX-License-Identifier: CC0-1.0
 */

#include <stdio.h>
#include <inttypes.h>
#include "sdkconfig.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_camera.h"
#include "esp_lcd_panel_io.h"

#define TAG "example"

void lcd_disp_init(void)
{
    ESP_LOGI(TAG, "Initialize Intel 8080 bus");
    esp_lcd_i80_bus_handle_t i80_bus = NULL;
    esp_lcd_i80_bus_config_t bus_config = {
        .clk_src = LCD_CLK_SRC_PLL240M,
        .dc_gpio_num = 0,
        .wr_gpio_num = 0,
        .data_gpio_nums = {
            0, 0, 0, 0, 0, 0, 0, 0
        },
        .bus_width = 8,
        .max_transfer_bytes = 200 * 100 * sizeof(uint16_t),
        .psram_trans_align = 64,
        .sram_trans_align = 4
    };
    ESP_ERROR_CHECK(esp_lcd_new_i80_bus(&bus_config, &i80_bus));

    ESP_LOGI(TAG, "Install panel IO");
    esp_lcd_panel_io_handle_t io_handle = NULL;
    esp_lcd_panel_io_i80_config_t io_config = {
        .cs_gpio_num        = 0,
        .pclk_hz            = 10000000,
        .trans_queue_depth  = 10,
        .dc_levels          = {
            .dc_idle_level  = 0,
            .dc_cmd_level   = 0,
            .dc_dummy_level = 0,
            .dc_data_level  = 1
        },
        .lcd_cmd_bits       = 8,
        .lcd_param_bits     = 8
    };
    ESP_ERROR_CHECK(esp_lcd_new_panel_io_i80(i80_bus, &io_config, &io_handle));
}

static camera_config_t camera_config = {
    .pin_pwdn       = 1,
    .pin_reset      = 1,
    .pin_xclk       = 1,
    .pin_sccb_sda   = 1,
    .pin_sccb_scl   = 2,

    .pin_pclk       = 1,
    .pin_href       = 1,
    .pin_vsync      = 1,
    .pin_d7         = 1,
    .pin_d6         = 1,
    .pin_d5         = 1,
    .pin_d4         = 1,
    .pin_d3         = 1,
    .pin_d2         = 1,
    .pin_d1         = 1,
    .pin_d0         = 1,

    .xclk_freq_hz   = 2000000,
    .ledc_timer     = 0,
    .ledc_channel   = 0,

    .pixel_format   = PIXFORMAT_RGB565,
    .frame_size     = FRAMESIZE_QVGA,

    .jpeg_quality   = 12,       //  0-63, for OV series camera sensors, lower number means higher quality
    .fb_count       = 1,        //When jpeg mode is used, if fb_count more than one, the driver will work in continuous mode.
    .grab_mode      = CAMERA_GRAB_WHEN_EMPTY,
};

void app_main(void)
{
    lcd_disp_init();
    ESP_ERROR_CHECK(esp_camera_init(&camera_config));
}

yes, I still can't see the issue with the above code. Please ignore the GPIO settings. I just want to test the esp_lcd_new_i80_bus and esp_camera_init can work at the same time because both of them will call esp_intr_alloc_intrstatus

liruya commented 8 months ago

LCD和Camera初始化冲突找到解决办法, 确实是配置中断的冲突, LCD和Camera中断用同一个中断源, 导致分配中断失败. 解决办法是LCD中断和Camera中断分配到不同的CPU. 比如LCD在CPU0 任务里初始化, Camera在CPU1任务里初始化. 另外发现必须先初始化Camera再初始化LCD, 否则还是有异常, 这个没发现什么原因.

目前LCD用80接口基本正常, 可以初始化成功, 摄像头实时采集并显示到LCD. 但是LCD用RGB接口, 可以初始化成功, 但是Camera采集失败, 提示Failed to get the frame on time!

suda-morris commented 8 months ago

Hi @liruya Thanks for keeping debug! The camera will allocate the interrupt with ESP_INTR_FLAG_IRAM. But this is not the case for the RGB LCD driver. In the RGB LCD driver, we don't use that flag unless the Kconfig LCD_RGB_ISR_IRAM_SAFE is enabled. That means, if you enabled that flag, then you can initialize the RGB LCD and camera driver from the same CPU core.

But I doubt if the camera should use ESP_INTR_FLAG_IRAM in the first place. I will ask the camera author to evaluate this.

AxelLin commented 7 months ago

@suda-morris

If my understanding is correct: gpio_install_isr_service() only takes effect for the first call. (The second call will return ESP_ERR_INVALID_STATE, "GPIO isr service already installed")

I'm wondering what happened if 2 components that both have called gpio_install_isr_service() with different intr_alloc_flags. Isn't this racy?

For example, if I have code using examples/bluetooth/esp_ble_mesh/common_components/button/button.c which calls gpio_install_isr_service(0); I also use esp32-camera driver which calls gpio_install_isr_service(ESP_INTR_FLAG_LOWMED | ESP_INTR_FLAG_IRAM);

Then different call sequence has different intr_alloc_flags setting. And if any of the component calls gpio_uninstall_isr_service(), both components stop working?

CC @me-no-dev for the question in https://github.com/espressif/esp-idf/issues/12882#issuecomment-1899873402

WangYuxin-esp commented 7 months ago

There are two areas worth investigating:

me-no-dev commented 7 months ago

@WangYuxin-esp thanks for the pointers!

@AxelLin please try https://github.com/espressif/esp32-camera/pull/629

me-no-dev commented 7 months ago

@suda-morris the flag is there to help speed things up a bit. ESP32 needs it to convert I2S samples to actual data. I can make it optional for S2 and S3 with LCD_CAM_ISR_IRAM_SAFE.

@AxelLin OK if I add it to the same PR for you to try?

AxelLin commented 6 months ago

@liruya Did you try esp32-camera v2.0.8? Is this still an issue?