esp32_camera fails when using esp-idf framework

[protectivedad]

The problem

Using esp-idf framework with esp32_camera causes the camera to fail. I found the solution but I don't know enough to properly implement it. The esp32_camera component hard codes version 1.0.0 for the espressif/esp32-camera library. This causes the problem. If I comment out and allow it to use the current version (2.0.4) it errors on the compile because of a problem which is detailed When using esp-idf, libraries downloaded by PlatformIO are not found, but there is workaround #281

This is where I don't know how to implement the solution the build directory CMakeLists.txt file needs:

list(APPEND EXTRA_COMPONENT_DIRS .piolibdeps/projectname)

added to it. After that it compiles and runs fine.

I haven't put a lot of tracing and debug information in this because the esp32_camera is using an old 1.0.0 library and probably should be updated.

Which version of ESPHome has the issue?

2023.10.3

What type of installation are you using?

pip

Which version of Home Assistant has the issue?

No response

What platform are you using?

ESP32-IDF

Board

esp32-cam

Component causing the issue

No response

Example YAML snippet

# Works:
esp32:
  board: esp32dev
  framework:
    type: arduino 

# Doesn't work:
esp32:
  board: esp32dev
  framework:
    type: esp-idf

Anything in the logs that might be useful for us?

[D][esp-idf:000]: I (179) sccb: pin_sda 26 pin_scl 27

[D][esp-idf:000]: I (180) gpio: GPIO[32]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 

[D][esp-idf:000]: I (219) sccb: SCCB_Probe start

[D][esp-idf:000]: E (711) camera: Detected camera not supported.

[D][esp-idf:000]: E (711) camera: Camera probe failed with error 0x20004

[E][esp32_camera:024]: esp_camera_init failed: ERROR
[E][component:113]: Component esp32_camera was marked as failed.

Additional information

No response

[protectivedad]

Spoke a little too soon. The camera compiles and the stream works but the snapshot mode fails:

[D][esp-idf:000]: E (8688) httpd: httpd_server_init: error in creating msg socket (23)

[E][component:113]: Component esp32_camera_web_server was marked as failed.

and later on:

[C][esp32_camera_web_server:088]: ESP32 Camera Web Server:
[C][esp32_camera_web_server:089]:   Port: 8080
[C][esp32_camera_web_server:091]:   Mode: stream
[C][esp32_camera_web_server:088]: ESP32 Camera Web Server:
[C][esp32_camera_web_server:089]:   Port: 8090
[C][esp32_camera_web_server:093]:   Mode: snapshot
[E][esp32_camera_web_server:097]:   Setup Failed
[D][esp32_camera:196]: Got Image: len=16858
[D][esp32_camera:196]: Got Image: len=22528
[D][esp32_camera:196]: Got Image: len=21141
[D][esp32_camera:196]: Got Image: len=21174
[D][esp32_camera:196]: Got Image: len=19358
[D][esp32_camera:196]: Got Image: len=19518
[D][esp32_camera:196]: Got Image: len=20354
[D][esp32_camera:196]: Got Image: len=20168
[D][esp32_camera:196]: Got Image: len=18767
[D][esp32_camera:196]: Got Image: len=19078
[D][esp32_camera:196]: Got Image: len=20154
[D][esp32_camera:196]: Got Image: len=20235

Which ever is first fails. Listing stream then snapshot -> snapshot fails. Listing snapshot then stream -> stream fails.

[protectivedad]

Alright that's solved. The reason I'm using esp-idf is to use tls mqtt. That seemed to use up the sockets so I increased the max sockets from 10 to 16. So my working yaml is:

packages:
  wifi: !include ../Default/wifi.yml
  ota: !include ../Default/ota.yml
  web_server: !include ../Default/web_server.yml
  mqtt: !include ../Default/mqtt.yml

esphome:
  name: test-esp32-cam-camera
  libraries:
    - espressif/esp32-camera

esp32:
  board: esp32dev
  framework:
    type: esp-idf
    sdkconfig_options:
      CONFIG_LWIP_MAX_SOCKETS: "16"

logger:

# Example configuration entry
esp32_camera:
  id: espcam
  external_clock:
    pin: GPIO0
    frequency: 10MHz
  i2c_pins:
    sda: GPIO26
    scl: GPIO27
  data_pins: [GPIO5, GPIO18, GPIO19, GPIO21, GPIO36, GPIO39, GPIO34, GPIO35]
  vsync_pin: GPIO25
  href_pin: GPIO23
  pixel_clock_pin: GPIO22
  power_down_pin: GPIO32

  # Image settings
  name: 'Test ESP32-Cam Camera'
  resolution: 1024x768
  jpeg_quality: 30

  brightness: 2 # -2 to 2
  contrast: 1 # -2 to 2
  special_effect: none
  # exposure settings
  aec_mode: auto
  aec2: false
  ae_level: 0
  aec_value: 300
  # gain settings
  agc_mode: auto
  agc_gain_ceiling: 2x
  agc_value: 0
  # white balance setting
  wb_mode: auto

output:
  - platform: ledc
    pin: GPIO4
    channel: 2
    id: led

light:
  - platform: monochromatic
    output: led
    name: 'Test ESP32-Cam Light'

esp32_camera_web_server:
  - port: 8081
    mode: snapshot
  - port: 8080
    mode: stream

with this patch to esphome:

--- a/__main__.py
+++ b/__main__.py
@@ -403,6 +403,12 @@ def command_compile(args, config):
     exit_code = write_cpp(config)
     if exit_code != 0:
         return exit_code
+
+    if CORE.using_esp_idf:
+        if os.path.islink(os.path.abspath(CORE.relative_build_path(""))+"/components"):
+            os.unlink(os.path.abspath(CORE.relative_build_path(""))+"/components")
+        os.symlink(".piolibdeps/"+CORE.name, os.path.abspath(CORE.relative_build_path(""))+"/components", True)
+
     if args.only_generate:
         _LOGGER.info("Successfully generated source code.")
         return 0
--- a/components/esp32_camera/__init__.py
+++ b/components/esp32_camera/__init__.py
@@ -278,10 +278,6 @@ async def to_code(config):

     cg.add_define("USE_ESP32_CAMERA")

-    if CORE.using_esp_idf:
-        cg.add_library("espressif/esp32-camera", "1.0.0")
-        add_idf_sdkconfig_option("CONFIG_RTCIO_SUPPORT_RTC_GPIO_DESC", True)
-
     for conf in config.get(CONF_ON_STREAM_START, []):
         trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var)
         await automation.build_automation(trigger, [], conf)

[dersch81]

I have the same issue to get an error of the camera when using it with esp-idf. I need it to use the esp32-cam as a bluetooth proxy what does not work with the arduino framework due to lack of memory. But i'm still failing to get it working with esp-idf.

How did you apply the patch? I'm using esphome inside a hassio-os docker.

[protectivedad]

The patch is just a hack a proof of concept. I wouldn't rely on it. but if you have to use this one:

--- a/__main__.py
+++ b/__main__.py
@@ -403,6 +403,16 @@ def command_compile(args, config):
     exit_code = write_cpp(config)
     if exit_code != 0:
         return exit_code
+
+    if CORE.using_esp_idf:
+        path = os.path.abspath(CORE.relative_build_path("")) + "/components"
+        link = path + "/esp32-camera"
+        if not os.path.exists(path):
+            os.makedirs(path)
+        if os.path.islink(link):
+            os.unlink(link)
+        os.symlink("../.piolibdeps/"+CORE.name+"/esp32-camera", link, True)
+
     if args.only_generate:
         _LOGGER.info("Successfully generated source code.")
         return 0
--- a/components/esp32_camera/__init__.py
+++ b/components/esp32_camera/__init__.py
@@ -278,10 +278,6 @@ async def to_code(config):

     cg.add_define("USE_ESP32_CAMERA")

-    if CORE.using_esp_idf:
-        cg.add_library("espressif/esp32-camera", "1.0.0")
-        add_idf_sdkconfig_option("CONFIG_RTCIO_SUPPORT_RTC_GPIO_DESC", True)
-
     for conf in config.get(CONF_ON_STREAM_START, []):
         trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var)
         await automation.build_automation(trigger, [], conf)

It's more likely to work with other things. The files are relative to the esphome directory. For me I have a venv ~/esphome-build/lib/python3.10/sitepackages/esphome I'm not familiar with hassio-os. Use quilt or something to be able to reverse the patch. It is not very friendly to rerunning, etc. I was hoping someone with more platformio experience would know how to properly insert the change.

[cyberlussi]

Just wanted to say, that I also stumbled over this when playing around with XIAO ESP32S3 Sense board. I really like to use esp-idf framework as it seems to generally run the system with a lower current consumption compared with arduino framework (~36mA vs. ~43mA) with identical basic esphome feature set i.e. Wifi, user LED). I started without the camera, both frameworks work fine, but when adding the camera module, esp-idf fails.

I tried to add platformio_options to modify the library version used via lib_deps to espressif/esp32-camera@^2.0.0, but it does not compile (.piolibdeps/xioa32s3sense-test/esp32-camera/conversions/to_jpg.cpp:29:2: error: #error Target CONFIG_IDF_TARGET is not supported). Using esphome libraries fails also, but with ValueError: Version pinning failed! Libraries espressif/esp32-camera@1.0.0 and espressif/esp32-camera@^2.0.0 requested with conflicting versions!.

So for the sake of being able to run battery powered esphome devices efficiently, esp-idf with espressif/esp32-camera@^2.0.0 should be possible.

[Rudd-O]

When is this going to be fixed?

Is there a non-patch workaround?

I will happily beta test anything that will allow this.

[DooMMasteR]

I am still seeing this happen...