Versions matching with framework-arduinoespressif32

[0x0fe]

I need Arduino ESP32 and the SDKConfig of IDF because a ton of option are changed for this project to run correcty, for this reason i use both platform-espressif32 and arduinoespressif32 (and IDF).

Previously i was on 6.3.2 and 2.0.11 with the following settings :

; 6.3.2 and 2.0.11
platform = https://github.com/platformio/platform-espressif32.git#v6.3.2
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#2.0.11
framework = 
    arduino
    espidf
board = esp32dev

But now I need some fixes in sdmmc https://github.com/espressif/arduino-esp32/issues/7373 which are available in IDF 5.1 and arduino ESP32 3.0, and more generally I want to get the latests fixes. I tried to move to 6.4.0 and 3.0.0-beta2

; 6.4.0 and 3.0.0-alpha2
platform = https://github.com/platformio/platform-espressif32.git#v6.4.0
platform_packages = framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#3.0.0-alpha2
framework = 
    arduino
    espidf
board = esp32dev

But PIO project configurator doesnt seem happy with that, can anyone tell how to set and which are the most recent versions compatible together?

fatal: Not a git repository (or any of the parent directories): .git
CMake Error at C:/Users/xxxxxx/.platformio/packages/framework-espidf@3.40405.230623/tools/cmake/build.cmake:463 (message):
  HINT: Please check manifest file of the following component(s):
  framework-arduinoespressif32-src-30ce9ca2702d9db78d435d2932a212dc

  ERROR: Because project depends on idf (>=5.1) which doesn't match any
  versions, version solving failed.

Then i tried without forcing the version of espressif32

platform = https://github.com/platformio/platform-espressif32.git
platform_packages = https://github.com/espressif/arduino-esp32.git#3.0.0-alpha2
framework = 
    arduino
    espidf
board = esp32dev

and i end up with this error, which i think is due to removing framework-arduinoespressif32 @ When i dd it back, i end up with the error " Because project depends on idf (>=5.1) which doesn't match any version"

Cloning into '/c/Users/xxxxxx/.platformio/.cache/tmp/pkg-installing-k5x07m5a'...

Platform Manager: espressif32@6.4.0+sha.ff6ec88 has been installed!
Tool Manager: Installing platformio/framework-arduinoespressif32 @ ~3.20011.0
Downloading 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Unpacking 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Tool Manager: framework-arduinoespressif32@3.20011.230801 has been installed!
Tool Manager: Installing arduino-esp32 @ *
UnknownPackageError: Could not find the package with 'arduino-esp32 @ *' requirements for your system 'windows_amd64'

[Jason2866]

@0x0fe This should work (not tested!):

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages = 
        framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git
        framework-espidf @ ~3.50101.0
framework = 
    arduino
    espidf
board = esp32dev

[mhaberler]

@Jason2866 tried your example - same result as @0x0fe

Processing m5stack-stamps3 (board: esp32dev; platform: https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5; framework: arduino, espidf)

Verbose mode can be enabled via -v, --verbose option CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html PLATFORM: Espressif 32 (2023.11.11+sha.1447bfb) > Espressif ESP32 Dev Module HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa) PACKAGES:

• framework-arduinoespressif32 @ 3.0.0+sha.bb72b02
• framework-espidf @ 3.50101.0 (5.1.1)
• tool-cmake @ 3.21.3
• tool-esptoolpy @ 1.40700.0+dev2 (4.7.0)
• tool-mklittlefs @ 3.2.0
• tool-mkspiffs @ 2.230.0 (2.30)
• tool-ninja @ 1.9.0
• tool-riscv32-esp-elf-gdb @ 12.1.0+20221002
• tool-xtensa-esp-elf-gdb @ 12.1.0+20221002
• toolchain-esp32ulp @ 1.23500.220830 (2.35.0)
• toolchain-xtensa-esp32 @ 12.2.0+20230208 Warning! Arduino framework as an ESP-IDF component doesn't handle the variant field! The default esp32 variant will be used. Reading CMake configuration... -- Found Git: /opt/homebrew/bin/git (found version "2.43.0") -- git rev-parse returned 'fatal: not a git repository (or any of the parent directories): .git' -- The C compiler identification is GNU 12.2.0 -- The CXX compiler identification is GNU 12.2.0 -- The ASM compiler identification is GNU -- Found assembler: /Users/mah/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Users/mah/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Users/mah/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- git rev-parse returned 'fatal: not a git repository (or any of the parent directories): .git' -- Could not use 'git describe' to determine PROJECT_VER. -- Building ESP-IDF components for target esp32 Skipping optional dependency: espressif/esp-sr Processing 19 dependencies: [1/19] chmorgan/esp-libhelix-mp3 (1.0.3) [2/19] espressif/cbor (0.6.0~1) [3/19] espressif/esp-dsp (1.4.9) [4/19] espressif/esp-zboss-lib (1.0.3) [5/19] espressif/esp-zigbee-lib (1.0.4) [6/19] espressif/esp_diag_data_store (1.0.1) [7/19] espressif/esp_diagnostics (1.0.1) [8/19] espressif/esp_insights (1.0.1) [9/19] espressif/esp_rainmaker (1.0.0) [10/19] espressif/esp_schedule (1.0.1) [11/19] espressif/esp_secure_cert_mgr (2.4.1) [12/19] espressif/jsmn (1.1.0) [13/19] espressif/json_generator (1.1.1) [14/19] espressif/json_parser (1.0.3) [15/19] espressif/mdns (1.2.1) [16/19] espressif/qrcode (0.1.0~1) [17/19] espressif/rmaker_common (1.4.3) [18/19] idf (5.1.1) [19/19] joltwallet/littlefs (1.10.2) Skipping optional dependency: espressif/esp-sr -- Configuring incomplete, errors occurred! See also "/Users/mah/Ballon/src/BalloonWare/idf/idf-arduino-test/.pio/build/m5stack-stamps3/CMakeFiles/CMakeOutput.log".

fatal: not a git repository (or any of the parent directories): .git CMake Error at /Users/mah/.platformio/packages/framework-espidf@src-16fd9488c8bbe3ed8a83c64ebfbb7d8c/tools/cmake/build.cmake:266 (message): Failed to resolve component 'spiffs'. Call Stack (most recent call first): /Users/mah/.platformio/packages/framework-espidf@src-16fd9488c8bbe3ed8a83c64ebfbb7d8c/tools/cmake/build.cmake:308 (build_resolve_and_add_req) /Users/mah/.platformio/packages/framework-espidf@src-16fd9488c8bbe3ed8a83c64ebfbb7d8c/tools/cmake/build.cmake:595 (build_expand_requirements) /Users/mah/.platformio/packages/framework-espidf@src-16fd9488c8bbe3ed8a83c64ebfbb7d8c/tools/cmake/project.cmake:547 (idf_build_process) CMakeLists.txt:3 (project)

[mhaberler]

taking out framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git makes your platformio.ini example work:

Resolving m5stack-stamps3 dependencies...
Platform espressif32 @ 2023.11.11+sha.1447bfb (required: git+https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5)
├── framework-arduinoespressif32 @ 3.0.0 (required: https://github.com/tasmota/arduino-esp32/releases/download/3.0.0.231118/framework-arduinoespressif32.zip)
├── framework-espidf @ 3.50101.0 (required: platformio/framework-espidf @ ~3.50101.0)
├── tool-cmake @ 3.21.3 (required: platformio/tool-cmake @ ~3.21.0)
├── tool-dfuutil-arduino @ 1.11.0 (required: platformio/tool-dfuutil-arduino @ ~1.11.0)
├── tool-esptoolpy @ 1.40700.0+dev2 (required: https://github.com/tasmota/esptool/releases/download/v4.7.0-dev2/esptool-4.7.0.zip)
├── tool-mkfatfs @ 2.0.1 (required: platformio/tool-mkfatfs @ ~2.0.0)
├── tool-mklittlefs @ 3.2.0 (required: tasmota/tool-mklittlefs @ ^3.2.0)
├── tool-mkspiffs @ 2.230.0 (required: platformio/tool-mkspiffs @ ~2.230.0)
├── tool-ninja @ 1.9.0 (required: platformio/tool-ninja @ ^1.7.0)
├── tool-openocd-esp32 @ 2.1100.20220706 (required: platformio/tool-openocd-esp32 @ ~2.1100.0)
├── tool-riscv32-esp-elf-gdb @ 12.1.0+20221002 (required: espressif/tool-riscv32-esp-elf-gdb @ ~12.1.0)
├── tool-xtensa-esp-elf-gdb @ 12.1.0+20221002 (required: espressif/tool-xtensa-esp-elf-gdb @ ~12.1.0)
├── toolchain-esp32ulp @ 1.23500.220830 (required: platformio/toolchain-esp32ulp @ ~1.23500.0)
├── toolchain-riscv32-esp @ 12.2.0+20230208 (required: espressif/toolchain-riscv32-esp @ 12.2.0+20230208)
└── toolchain-xtensa-esp32s3 @ 12.2.0+20230208 (required: espressif/toolchain-xtensa-esp32s3 @ 12.2.0+20230208)

[0x0fe]

@mhaberler well, sure, but then you dont have access the menuconfig, which is the whole point.

[mhaberler]

ah, because that's a source tree, not a precompiled archive?

[Jason2866]

Mhh, looks like it is still using my IDF build. Not at my PC so I can't test. Try this one which adds the owner of the IDF build

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages = 
        framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git
        platformio/framework-espidf @ ~3.50101.0
framework = 
    arduino
    espidf
board = esp32dev

If this does not solve delete the hidden folder .platformio There are probably leftovers which do harm.

[0x0fe]

@mhaberler yes, i think so, adding this is what allowed me to keep the menuconfig previously. @Jason2866 yes, just tried, it went into a strange error, i will retry.

  File "C:\Users\xxxxxx\.platformio\python3\lib\os.py", line 225, in makedirs
    mkdir(name, mode)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\xxxxxx\\.platformio\\packages\\framework-arduinoespressif32'

============================================================

An unexpected error occurred. Further steps:

[mhaberler]

@Jason2866 tried your suggestion, wiped .platformio, let re-download:

[env:m5stack-stamps3]
board = m5stack-stamps3
build_type = debug 
debug_tool = esp-builtin

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages = 
        framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git
        platformio/framework-espidf @ ~3.50101.0

framework = 
    arduino
    espidf

resulted in:

 Executing task: platformio run --environment m5stack-stamps3 

Processing m5stack-stamps3 (board: m5stack-stamps3; platform: https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5; framework: arduino, espidf)
----------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/m5stack-stamps3.html
PLATFORM: Espressif 32 (2023.11.11+sha.1447bfb) > M5Stack StampS3
HARDWARE: ESP32S3 240MHz, 320KB RAM, 8MB Flash
DEBUG: Current (esp-builtin) External (cmsis-dap, esp-bridge, esp-builtin, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.0.0+sha.bb72b02 
 - framework-espidf @ 3.50101.230828 (5.1.1) 
 - tool-cmake @ 3.21.3 
 - tool-esptoolpy @ 1.40700.0+dev2 (4.7.0) 
 - tool-mklittlefs @ 3.2.0 
 - tool-mkspiffs @ 2.230.0 (2.30) 
 - tool-ninja @ 1.9.0 
 - tool-riscv32-esp-elf-gdb @ 12.1.0+20221002 
 - tool-xtensa-esp-elf-gdb @ 12.1.0+20221002 
 - toolchain-esp32ulp @ 1.23500.220830 (2.35.0) 
 - toolchain-riscv32-esp @ 12.2.0+20230208 
 - toolchain-xtensa-esp32s3 @ 12.2.0+20230208
Warning! Arduino framework as an ESP-IDF component doesn't handle the `variant` field! The default `esp32` variant will be used.
Reading CMake configuration...

*** Two environments with different actions were specified for the same target: /Users/mah/Ballon/src/BalloonWare/idf/idf-arduino-test/.pio/build/m5stack-stamps3/managed_components/espressif__esp-dsp/modules/support/cplx_gen/dsps_cplx_gen.o
(action 1: xtensa-esp32s3-elf-gcc -g2 -ggdb2 -DPLATFORMIO=60112 -DARDUINO_M5Stack_StampS3 -DARDUINO_USB_MODE=1 -DARDUINO_RUNNING_CORE=1 -DARDUINO_EVENT_RUNNING_CORE=1 -DESP_PLATFORM -DIDF_VER=\"5.1.1\" -DSOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE -D_GNU_SOURCE -D_POSIX_READER_WRITER_LOCKS -DconfigENABLE_FREERTOS_DEBUG_OCDAWARE=1 -D__PLATFORMIO_BUILD_DEBUG__ -c -o dsps_cplx_gen.o dsps_cplx_gen.S)
(action 2: xtensa-esp32s3-elf-gcc -mlongcalls -ffunction-sections -fdata-sections -Wall -Werror=all -Wno-error=unused-function -Wno-error=unused-variable -Wno-error=unused-but-set-variable -Wno-error=deprecated-declarations -Wextra -Wno-unused-parameter -Wno-sign-compare -Wno-enum-conversion -gdwarf-4 -ggdb -Og -fmacro-prefix-map=/Users/mah/Ballon/src/BalloonWare/idf/idf-arduino-test=. -fmacro-prefix-map=/Users/mah/.platformio/packages/framework-espidf=/IDF -fstrict-volatile-bitfields -fno-jump-tables -fno-tree-switch-conversion -g2 -ggdb2 -DPLATFORMIO=60112 -DARDUINO_M5Stack_StampS3 -DARDUINO_USB_MODE=1 -DARDUINO_RUNNING_CORE=1 -DARDUINO_EVENT_RUNNING_CORE=1 -DESP_PLATFORM -DIDF_VER=\"5.1.1\" -DSOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE -D_GNU_SOURCE -D_POSIX_READER_WRITER_LOCKS -DconfigENABLE_FREERTOS_DEBUG_OCDAWARE=1 -D__PLATFORMIO_BUILD_DEBUG__ -c -o dsps_cplx_gen.o dsps_cplx_gen.S)
File "/Users/mah/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 690, in compile_source_files

[Jason2866]

@mhaberler Progress, this looks like a pio setup issue in your project. The Pio compile setup looks good so far.

[mhaberler]

@Jason2866 not sure about that, other projects build fine

I applied the same platformio.ini to https://github.com/platformio/platform-espressif32/tree/develop/examples/espidf-arduino-wifiscan to start with a fresh repo

same thing - error message travels with your ini

[Jason2866]

@mhaberler This makes the issue https://github.com/espressif/arduino-esp32/blob/master/idf_component.yml Replace the yml file content in your folder .platformio/packages/framework-arduinoespressif32/idf_component.yml with this content

description: "Arduino core for ESP32, ESP32-S and ESP32-C series of SoCs"
url: "https://github.com/espressif/arduino-esp32"
targets:
  - esp32
  - esp32s2
  - esp32s3
  - esp32c2
  - esp32c3
  - esp32c6
  - esp32h2
tags:
  - arduino
files:
  include:
    - "cores/**/*"
    - "variants/esp32/**/*"
    - "variants/esp32s2/**/*"
    - "variants/esp32s3/**/*"
    - "variants/esp32c2/**/*"
    - "variants/esp32c3/**/*"
    - "variants/esp32c6/**/*"
    - "variants/esp32h2/**/*"
    - "libraries/**/*"
    - "CMakeLists.txt"
    - "Kconfig.projbuild"
  exclude:
    - "**/*"
dependencies:
  idf: ">=5.1"
  # mdns 1.2.1 is necessary to build H2 with no WiFi
  mdns: "1.2.1"
  joltwallet/littlefs: "^1.10.2"

The dsp dependence in the orig file is conflicting in the setup.

[Jason2866]

With this change i can compile a arduino / idf project. But in IDF 5.1 the RMT part is too old for actual Arduino code

[0x0fe]

trying that then, still unclear to me why i had to manually add (an old) SPIFFS into the IDF directory when using your esp32core, i dont know how the esp32core can impact the IDF directory

[Jason2866]

@0x0fe It is listed here as dependence https://github.com/espressif/arduino-esp32/blob/master/CMakeLists.txt Remove all entries with SPIFFS and it will compile

[0x0fe]

@Jason2866 I see, i did that, but i dont know how you could compile, here it fails due to API differences with RMT, apparently.

With this ini

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages = 
        framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git
        platformio/framework-espidf @ ~3.50101.0

framework = 
    arduino
    espidf

and the modified yml

Compiling .pio\build\esp32dev\FunctionalInterrupt.o
C:/Users/xxxxx/.platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-rmt.c: In function 'rmtInit':
C:/Users/xxxxx/.platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-rmt.c:516:11: error: 'rmt_tx_channel_config_t' has no member named 'intr_priority'
  516 |     tx_cfg.intr_priority = 0;
      |           ^
C:/Users/xxxxx/.platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal-rmt.c:541:11: error: 'rmt_rx_channel_config_t' has no member named 'intr_priority'
  541 |     rx_cfg.intr_priority = 0;
      |           ^
Compiling .pio\build\esp32dev\HardwareSerial.o
*** [.pio\build\esp32dev\esp32-hal-rmt.o] Error 1

Should i force it to #3.0.0-alpha2?

[0x0fe]

so, indeed forcing to 3.0.0-alpha2 allowed me to build without error

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages = 
        framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#3.0.0-alpha2
        framework-espidf @ ~3.50101.0
framework = 
    arduino
    espidf
board = esp32dev

Building .pio\build\esp32dev\bootloader.bin
Compiling .pio\build\esp32dev\libd22\MQTT\lwmqtt\helpers.c.o
esptool.py v4.7-dev
Creating esp32 image...
Merged 1 ELF section
Successfully created esp32 image.

[mhaberler]

@0x0fe worked for me as well

until the next snag comes around - let me take a step back:

what's the best approach here short of patching up things under .platformio which might live only a few days?

should I just fork the Jason2866/platform-espressif32 and espressif/arduino-esp32 repos and work with the source-only variants?

[Jason2866]

Currently the only way to use official sources is not using the latest Arduino code from branch master. The alpha2 build is a bit outdated already, but has all major changes already "in". When Platformio releases the actual IDF 5.1.2 the Arduino master branch will compile.

[mhaberler]

thanks!

[0x0fe]

build works fine however when i inlcuded FS, though part of the arduinoespressif32 "framework", i get this vtable error, which to me is like a CPP linkage error. Does anyone have an idea of the cause? I it related to the SPIFFS issue?

|-- FS @ 2.0.0
|-- SD_MMC @ 2.0.0
|-- WiFi @ 2.0.0
Building in release mode
Compiling .pio\build\esp32dev\src\main.o
Linking .pio\build\esp32dev\firmware.elf
c:/users/xxxxx/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/12.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32dev\esp-idf\framework-arduinoespressif32-
src-30ce9ca2702d9db78d435d2932a212dc\libframework-arduinoespressif32-src-30ce9ca2702d9db78d435d2932a212dc.a(vfs_api.o):(.literal._ZN11VFSFileImplD2Ev+0x4): undefined reference to `vtable for fs::
FileImpl'
collect2.exe: error: ld returned 1 exit status
*** [.pio\build\esp32dev\firmware.elf] Error 1
=================================================================================== [FAILED] Took 17.51 seconds 

[Jason2866]

It is a bug in Arduino framework, solved after alpha2. You can avoid if you add sdkconfig.defaults with

CONFIG_AUTOSTART_ARDUINO=y
# CONFIG_WS2812_LED_ENABLE is not set
CONFIG_FREERTOS_HZ=1000

CONFIG_BOOTLOADER_COMPILER_OPTIMIZATION_SIZE=y
CONFIG_COMPILER_OPTIMIZATION_SIZE=y

This commit fixes this error https://github.com/espressif/arduino-esp32/commit/4b3aebe11fa5acc0f8b259105b4d1990b95c399b

[0x0fe]

@Jason2866 oh nice, thanks, i am lucky you knew it already. EDIT : it was the problem indeed

[Jason2866]

@mhaberler @0x0fe This is using orig IDF v5.1.2 and latest Arduino master (removed the clashing sr and dsp libs)

[env]
platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages =
        framework-arduinoespressif32 @ https://github.com/tasmota/arduino-esp32.git#master
        framework-espidf @ https://github.com/tasmota/esp-idf/releases/download/v5.1.2-org/esp-idf-v5.1.2-org.zip
framework = arduino, espidf

build_flags =
    -D CONFIG_BLINK_GPIO=2
monitor_speed = 115200

[env:esp32dev]
board = esp32dev

[env:esp32-c2-devkitm-1]
board = esp32-c2-devkitm-1

[env:esp32-c6-devkitm-1]
board = esp32-c6-devkitm-1

For Arduino IDF projects you have to configure a valid sdkconfig. But this is usual business

[0x0fe]

@Jason2866 nice, i will try it in a separate project. By the way, did you encouter issue with PSRAM? as soon as i enabled it i get some includes errors

Compiling .pio\build\esp32dev\esp32-hal-time.o
Compiling .pio\build\esp32dev\esp32-hal-timer.o
Compiling .pio\build\esp32dev\esp32-hal-tinyusb.o
C:/Users/xxxxx/.platformio/packages/framework-arduinoespressif32-src-30ce9ca27
sram.c:22:10: fatal error: esp_psram.h: No such file or directory

[Jason2866]

Thats tricky in every Arduino / IDF project. The Arduino Lib builder does copy some files in specific folders in the built Arduino framework. This does not happen when building Arduino as IDF component. You have to search the missing file(s) and provide it "somehow"

That are the python scripts in the Arduino Lib builder for that https://github.com/espressif/esp32-arduino-lib-builder/blob/master/tools/copy-mem-variant.sh https://github.com/espressif/esp32-arduino-lib-builder/blob/master/tools/copy-libs.sh

[Jason2866]

@Jason2866 nice, i will try it in a separate project. By the way, did you encouter issue with PSRAM? as soon as i enabled it i get some includes errors

Compiling .pio\build\esp32dev\esp32-hal-time.o
Compiling .pio\build\esp32dev\esp32-hal-timer.o
Compiling .pio\build\esp32dev\esp32-hal-tinyusb.o
C:/Users/xxxxx/.platformio/packages/framework-arduinoespressif32-src-30ce9ca27
sram.c:22:10: fatal error: esp_psram.h: No such file or directory

@0x0fe or better open an issue in espressif Arduino github. Compiling Arduino as component is officially supported. The error you encounter is NOT caused by using Platformio.

[valeros]

Support for Arduino 3.0 is tracked in https://github.com/platformio/platform-espressif32/issues/1225

[peteDDD]

Happily got running using @Jason2866 advice above ... platformio.ini

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
platform_packages =
      framework-arduinoespressif32 @ https://github.com/tasmota/arduino-esp32.git#master
      framework-espidf @ https://github.com/tasmota/esp-idf/releases/download/v5.1.2-org/esp-idf-v5.1.2-org.zip
framework = arduino, espidf

But can't seem to turn off ESP-Insights and ESP-Rainmaker which both want compile mqtt_server.crt or https_server.crt which require the respective .S file that is never found. I have tried to turn off Insights and Rainmaker (and for that matter everything that uses https) using menuconfig but I still fail to build ...

...
Compiling .pio\build\esp32-s3-devkitc-1\managed_components\espressif__esp_insights\src\esp_insights.o
Compiling .pio\build\esp32-s3-devkitc-1\managed_components\espressif__esp_insights\src\esp_insights_transport.o
Compiling .pio\build\esp32-s3-devkitc-1\managed_components\espressif__esp_insights\src\esp_insights_client_data.o
Compiling .pio\build\esp32-s3-devkitc-1\managed_components\espressif__esp_insights\src\esp_insights_encoder.o
Compiling .pio\build\esp32-s3-devkitc-1\managed_components\espressif__esp_insights\src\esp_insights_cbor_decoder.o
Compiling .pio\build\esp32-s3-devkitc-1\managed_components\espressif__esp_insights\src\esp_insights_cbor_encoder.o
.pio\build\esp32-s3-devkitc-1\mqtt_server.crt.S' not found, needed by target `.pio\build\esp32-s3-devkitc-1\.pio\build\esp32-s3-devkitc-1\mqtt_server.crt.o'.

Any wisdom on how to get these "managed componets" out of my life?

[Jason2866]

@peteDDD Thats the main change in my fork. Removed that stuff you mentioned. So you can do

platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
framework = arduino, espidf

[Jason2866]

@mhaberler @peteDDD @0x0fe I have made a fork using official espressif repos and libs. Verified working Arduino, IDF and Arduino as a component from IDF. For Arduino as an component of IDF see espidf-arduino-blink

[env:development]
platform = https://github.com/jason2866/platform-espressif32.git#Arduino/IDF5_org

board = ...
...

Arduino firmwares built with the official sources and settings will be much bigger in size compared to the Tasmota fork.

[mhaberler]

@Jason2866 appreciated, thanks!

tried with a few projects, seems to have an issue with Nimble - where should I report issues?

https://github.com/mhaberler/nimble-bl5-examples/commit/1126ddb581cec0391620c1adba96f6835065101d https://github.com/mhaberler/arduino3-playground/commit/0d0a97e076456d5206646cadbf50283491a6c8b1

these used to work with Arduino/IDF5

edit: please see the top commits in the first link for working/broken combinations

[Jason2866]

@mhaberler Did not say that the orig. espressif Arduino setup is a good one ;-) h2zero NimBLE does work perfectly fine with Tasmota Arduino 3.0.0 We use https://github.com/h2zero/esp-nimble-cpp since it supports C6 and C2 already. The Arduino version does not support the new MCUs. Maybe has issues with Arduino 3.0.0 in general.

To answer your question. Since it is not an released version. Nowhere. Use the one which works for you.

[newmansnj]

@Jason2866 nice, i will try it in a separate project. By the way, did you encouter issue with PSRAM? as soon as i enabled it i get some includes errors

Compiling .pio\build\esp32dev\esp32-hal-time.o
Compiling .pio\build\esp32dev\esp32-hal-timer.o
Compiling .pio\build\esp32dev\esp32-hal-tinyusb.o
C:/Users/xxxxx/.platformio/packages/framework-arduinoespressif32-src-30ce9ca27
sram.c:22:10: fatal error: esp_psram.h: No such file or directory

have you solve that problem or found a Workaround ?

[Jason2866]

@newmansnj No issue example does compile successfully. PSRAM ia activated

[newmansnj]

@newmansnj No issue example does compile successfully. PSRAM ia activated

I have used this sdkconfig and become with this (and also the standart ) board json file the same psram error. json: { "build": { "arduino":{ "ldscript": "esp32s3_out.ld", "partitions": "default_16MB.csv", "memory_type": "qio_opi" }, "core": "esp32", "extra_flags": [ "-DARDUINO_ESP32S3_DEV", "-DBOARD_HAS_PSRAM", "-DARDUINO_USB_MODE=1", "-DARDUINO_USB_CDC_ON_BOOT=1" ], "f_cpu": "240000000L", "f_flash": "80000000L", "flash_mode": "qio", "psram_type": "opi", "hwids": [ [ "0x303A", "0x1001" ] ], "mcu": "esp32s3", "variant": "esp32s3" }, "connectivity": [ "wifi", "bluetooth" ], "debug": { "default_tool": "esp-builtin", "onboard_tools": [ "esp-builtin" ], "openocd_target": "esp32s3.cfg" }, "frameworks": [ "arduino", "espidf" ], "name": "Espressif ESP32-S3-DevKitC-1-N16R8V (16 MB QD, 8MB PSRAM)", "upload": { "flash_size": "16MB", "maximum_ram_size": 327680, "maximum_size": 16777216, "require_upload_port": true, "speed": 921600 }, "url": "https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/esp32s3/user-guide-devkitc-1.html", "vendor": "Espressif" } sdkonfig.esp32-s3: (sorry for //# but i dont now how to show it in right formart on github) `. . . //# //# ESP PSRAM //# CONFIG_SPIRAM=y

CONFIG_SPIRAM_MODE_OCT=y CONFIG_SPIRAM_TYPE_AUTO=y //# CONFIG_SPIRAM_TYPE_ESPPSRAM64 is not set CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y CONFIG_SPIRAM_CLK_IO=30 CONFIG_SPIRAM_CS_IO=26 //# CONFIG_SPIRAM_FETCH_INSTRUCTIONS is not set //# CONFIG_SPIRAM_RODATA is not set //# CONFIG_SPIRAM_SPEED_80M is not set CONFIG_SPIRAM_SPEED_40M=y CONFIG_SPIRAM_SPEED=40 CONFIG_SPIRAM_BOOT_INIT=y //# CONFIG_SPIRAM_IGNORE_NOTFOUND is not set //# CONFIG_SPIRAM_USE_MEMMAP is not set //# CONFIG_SPIRAM_USE_CAPS_ALLOC is not set CONFIG_SPIRAM_USE_MALLOC=y CONFIG_SPIRAM_MEMTEST=y CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=64000 CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=65000 //# CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY is not set //# CONFIG_SPIRAM_ECC_ENABLE is not set //# end of SPI RAM config //# end of ESP PSRAM . . .`

platformio.ini `[platformio] src_dir = main

[env:esp32-s3] platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5 ; platform_packages = ; framework-arduinoespressif32 @ https://github.com/tasmota/arduino-esp32.git#master ; framework-espidf @ https://github.com/tasmota/esp-idf/releases/download/v5.1.2-org/esp-idf-v5.1.2-org.zip framework = arduino, espidf ; platform = https://github.com/tasmota/platform-espressif32/releases/download/2024.02.10/platform-espressif32.zip ;platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5

; platform = file:///home/jens/Desktop/framework-espidf_jens ; framework = arduino, espidf ; platform_packages = tool-xtensa-esp-elf-gdb upload_speed = 2000000 ; upload_port = COM23 monitor_speed = 115200

board = esp32-s3-devkitc-1-n16r8v ; board_build.arduino.memory_type = qio_opi ; board_upload.flash_size = 16MB ; board_build.flash_mode = dio

lib_deps = mikalhart/TinyGPSPlus@^1.0.3 google/googletest@^1.12.1 build_flags =

-D ARDUINO_LMIC_PROJECT_CONFIG_H_SUPPRESS
-D CFG_eu868=1
-D LMIC_DEBUG_LEVEL=0
-D CFG_sx1276_radio=1
-D DISABLE_MCMD_DutyCycleReq
-D hal_init=LMICHAL_init
;   -I /home/jens/.platformio/packages/framework-arduinoespressif32-src-679bfc94fa27bca81f4e53d483fd06c1/tools/esp32-arduino-libs/esp32s3/include/esp_psram/include`

[Jason2866]

@newmansnj Sorry with this fragments this leads to nowhere. I can have a look if you provide a not to complex project on github where the problem arrives.

[newmansnj]

@newmansnj Sorry with this fragments this leads to nowhere. I can have a look if you provide a not to complex project on github where the problem arrives.

Ok i will provide a less complex project .

[newmansnj]

Ok i found out when you have in Platofrmio.ini ` platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5 upload_speed = 2000000 monitor_speed = 115200 board = esp32-s3-devkitc-1 build_flags = -D BOARD_HAS_PSRAM

` With the build_flags :

build_flags = -D BOARD_HAS_PSRAM in it the error "Compiling .pio/build/esp32-s3/esp32-hal-sigmadelta.o /home/jens/.platformio/packages/framework-arduinoespressif32-src-679bfc94fa27bca81f4e53d483fd06c1/cores/esp32/esp32-hal-psram.c:22:10: fatal error: esp_psram.h: No such file or directory"

With this build flag the error occurs without the build flag the error does not occur. It is even built

Compiling .pio/build/esp32-s3/esp32-hal-misc.o Compiling .pio/build/esp32-s3/esp32-hal-periman.o Compiling .pio/build/esp32-s3/esp32-hal-psram.o Compiling .pio/build/esp32-s3/esp32-hal-rgb-led.o Compiling .pio/build/esp32-s3/esp32-hal-sigmadelta.o

However, in many boad .json files this command also appears. "extra_flags": [ "-DARDUINO_ESP32S3_DEV", "-DBOARD_HAS_PSRAM", "-DARDUINO_USB_MODE=1", "-DARDUINO_USB_CDC_ON_BOOT=1" ],

[Jason2866]

@newmansnj This is okay, but when using Arduino as a component from IDF this has to be set in sdkconfig file too. Without the IDF part is not configured to use PSRAM. So the needed files will be not generated. There is no "reverse translating" from this setting to the sdkconfig.

So pio run -t menuconfig is your way to solve this.

[newmansnj]

@newmansnj This is okay, but when using Arduino as a component from IDF this has to be set in sdkconfig file too. Without the IDF part is not configured to use PSRAM. So the needed files will be not generated. There is no "reverse translating" from this setting to the sdkconfig.

So pio run -t menuconfig is your way to solve this.

I have usded also pio run -t menuconfig for flash and psram (and many other things to config) it dosent change the outcome of this problem.

[newmansnj]

I have this config: CONFIG_SPIRAM=y

CONFIG_SPIRAM_MODE_OCT=y CONFIG_SPIRAM_TYPE_AUTO=y

CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y CONFIG_SPIRAM_CLK_IO=30 CONFIG_SPIRAM_CS_IO=26 CONFIG_SPIRAM_SPEED_40M=y CONFIG_SPIRAM_SPEED=40 CONFIG_SPIRAM_BOOT_INIT=y CONFIG_SPIRAM_USE_MALLOC=y CONFIG_SPIRAM_MEMTEST=y CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=64000 CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=65000

and: build_flags = -D BOARD_HAS_PSRAM

writen in the files (ini and sdkconfig)

[Jason2866]

@newmansnj Can reproduce now. The file esp_psram.h is not added as include file when building with Platformio. The CMakeLists.txt https://github.com/tasmota/esp-idf/blob/release/v5.1/components/esp_psram/CMakeLists.txt is adding it.

idf_component_register(SRCS ${srcs}
                       INCLUDE_DIRS ${includes}
                       PRIV_REQUIRES ${priv_requires}
                       LDFRAGMENTS linker.lf)

Don't know what's going wrong here. Looks like a very special issue from Platformio.

[newmansnj]

Thank you 👍 There are many other things that have problems with the new Alpha Arduino 3.0 (but I'm not sure whether Platformio or esp-idf is to blame). Like for example that with the new version (although I had explicitly switched on menueconfig and rechecked it) no check of the psram is done after the first compile you can see that in the output.

There are also problems with flash. I always get SHA-256 comparison failed: Calculated: 4cf531339cf89e30c717a5d214c8c9b4327d470fb3d9d86fd0c771bfd56c8655 Expected: fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff Attempting to boot anyway... (but these are just a few of many errors and I don't know if Platformio or ESP-idf is to blame or for some it's my fault) but I just wanted to point it out here if not already known.

[Jason2866]

The checksum calculation is by design off in my implementation. It is useless when the firmware is patched by esptool when flashing outside the IDE. The check is hard coded and can't be switched off. So no bug just an info that's no checksum is there

[Jason2866]

@newmansnj To get a successful compile when PSRAM is enabled add as workaround to your env:

build_flags =
    -I"${platformio.packages_dir}/framework-espidf/components/esp_psram/include"

[newmansnj]

I also have problems debugging it at the moment. First it said : "Compiling .pio/build/esp32-s3-devkitc-1/esp_hw_support/port/esp32s3/rtc_sleep.o Assembler messages: Fatal error: can't create .pio/build/esp32-s3-devkitc-1/esp_hw_support/port/esp32s3/rtc_clk_init.o: Datei oder Verzeichnis nicht gefunden *** [.pio/build/esp32-s3-devkitc-1/esp_hw_support/port/esp32s3/rtc_clk_init.o] Error 1 In file included from /home/jens/.platformio/packages/framework-espidf@src-6b823ff451cbc090798fd415ab24115d/components/esp_hw_support/port/esp32s3/rtc_sleep.c:8: /home/jens/.platformio/packages/framework-espidf@src-6b823ff451cbc090798fd415ab24115d/components/esp_common/include/esp_attr.h:14:10: fatal error: sdkconfig.h: No such file or directory

---

• Looking for sdkconfig.h dependency? Check our library registry!
• CLI > platformio lib search "header:sdkconfig.h"
• Web > https://registry.platformio.org/search?q=header:sdkconfig.h

• ---

  14 | #include "sdkconfig.h" | ^~~~~ compilation terminated. *** [.pio/build/esp32-s3-devkitc-1/esp_hw_support/port/esp32s3/rtc_sleep.o] Error 1"