Creating a customized framework for the ESP-IDF results in a Python error

MrSurly commented 3 weeks ago

What kind of issue is this?

[X] PlatformIO Core. If you’ve found a bug, please provide an information below.

Configuration

Operating system: Ubuntu Linux 22.04

PlatformIO Version (platformio --version): PlatformIO Core, version 6.1.15

Description of problem

N.B.: This was originally a forum post, but that got zero traction.

I have a custom framework (based on ESP32 IDF) being pulled from a repository, named esp-idf-ssr. It’s defined in my platformio.ini file (see below). I also have a custom board located in boards/tc.json in the top level of my project.

The tc board does show up for the espressif32 platform when you issue pio boards.

The tc.json file is identical as the esp32dev board, except I added the custom framework.

Problems:

Custom framework with stock board:

When using espidf-ssr framework with esp32dev board:

Error: This board doesn't support espidf-ssr framework!

Custom framework with custom board:

When using espidf-ssr framework with tc board:

Processing TC (platform: espressif32 @ 6.5.0; framework: espidf-ssr; board: tc)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/tc.html
PLATFORM: Espressif 32 (6.5.0) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-espidf-ssr @ 3.50201.0+sha.70c9b04 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
AssertionError: :
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/main.py", line 346:
    target_elf = env.BuildProgram()
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 347:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/pioplatform.py", line 58:
    assert p.frameworks and framework in p.frameworks
============================================================================== [FAILED] Took 0.15 seconds ==============================================================================

Environment    Status    Duration
-------------  --------  ------------
TC             FAILED    00:00:00.149
========================================================================= 1 failed, 0 succeeded in 00:00:00.149 =========================================================================

Stock framework, but override framework-espidf with custom URL:

platformio.ini snippets:

platform = espressif32 @ 6.5.0
platform_packages = framework-espidf@ <redacted>/esp-idf-ssr.git#v5.2.1-ssr
framework = espidf
board = esp32dev

Here, I’m simply trying to override the existing framework-espidf for use with the esp32dev board. I get:

TypeError: can only concatenate str (not "NoneType") to str:
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/main.py", line 346:
    target_elf = env.BuildProgram()
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Util/envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "/home/epoulsen/.platformio/penv/lib/python3.10/site-packages/platformio/builder/tools/piobuild.py", line 347:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 676:
    return method(*args, **kw)
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/epoulsen/.platformio/packages/tool-scons/scons-local-4.7.0/SCons/Script/SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 1251:
    ensure_python_venv_available()
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 1224:
    venv_dir = get_idf_venv_dir()
  File "/home/epoulsen/.platformio/platforms/espressif32/builder/frameworks/espidf.py", line 1178:
    env.subst("$PROJECT_CORE_DIR"), "penv", ".espidf-" + idf_version

How can I use my custom framework? I only added the custom board thinking I needed to do that to avoid the This board doesn't support espidf-ssr framework! error

Update 1

I found what’s causing this (sorta)

In [platform]/espressif32/builder/frameworks/espidf.py function get_idf_venv_dir, the call to get_original_version fails because the version is 3.50201.0+sha.8ef99ab. It has the +sha... tacked onto the end, and get_original_version won’t parse it because it has 3 dots (.) in it:

def get_original_version(version):
    if version.count(".") != 2:
        return None
    _, raw = version.split(".")[:2]
    if int(raw) <= 99:
        return None
    if int(raw) <= 9999:
        return "%s.%s" % (raw[:-2], int(raw[-2:]))
    return "%s.%s.%s" % (raw[:-4], int(raw[-4:-2]), int(raw[-2:]))

Update 2

The version data is ultimately coming from .../packages/framework-espidf/.git/.piopm But I cannot figure out what writes this file. I did a search for the string .piopm in the ~/.platformio directory – got a few hits, but nothing related to what creates this file.

Steps to Reproduce

[Detailed description above]

Actual Results

Crashes with Python assertion

Expected Results

To work as a custom framework

If problems with PlatformIO Build System:

The content of platformio.ini:

[env] ## Defaults
monitor_speed = 115200
platform = espressif32 @ 6.5.0
platform_packages = 
    framework-espidf-ssr @ git@<redacted>/esp-idf-ssr.git#v5.2.1-ssr
framework = espidf-ssr

board = tc

Source file to reproduce issue:

N/A

Szybet commented 1 week ago

I also have this issue ;/

Szybet commented 1 week ago

The workaround for me is just to point to a release zip file for example

gojimmypi commented 2 days ago

I have a similar issue. Here's my platformio.ini file:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = espidf
monitor_speed = 115200

platform_packages = 
    framework-espidf @ file:///SysGCC/esp32/esp-idf/v5.2

The ESP-IDF in that directory is known to be good, and works with both the idf.py command, as well as from VisualGDB.

When I delete all the related directories in C:\Users\gojimmypi\.platformio\.cache and and C:\Users\gojimmypi\.platformio\packages\framework-espidf to install fresh, this is the error I see:

pio run
Processing esp32dev (platform: espressif32; board: esp32dev; framework: espidf)
------------------------------------------------------------------------------------------------------------------------
Tool Manager: Installing platformio/toolchain-xtensa-esp-elf @ 13.2.0+20230928
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: toolchain-xtensa-esp-elf@13.2.0+20230928 has been installed!
Tool Manager: Installing espressif/tool-xtensa-esp-elf-gdb @ ~11.2.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-xtensa-esp-elf-gdb@11.2.0+20230208 has been installed!
Tool Manager: Installing espressif/tool-riscv32-esp-elf-gdb @ ~11.2.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-riscv32-esp-elf-gdb@11.2.0+20220823 has been installed!
Tool Manager: Installing file:///SysGCC/esp32/esp-idf/v5.2
Tool Manager: framework-espidf@0.0.0+20240630140639 has been installed!
Tool Manager: Resolving dependencies...
AssertionError: Traceback (most recent call last):
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\__main__.py", line 103, in main
    cli()  # pylint: disable=no-value-for-parameter
    ^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\cli.py", line 85, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\run\cli.py", line 145, in cli
    process_env(
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\run\cli.py", line 208, in process_env
    ).process()
      ^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\run\processor.py", line 81, in process
    install_project_env_dependencies(
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\commands\install.py", line 132, in install_project_env_dependencies
    _install_project_env_platform(project_env, options),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\commands\install.py", line 149, in _install_project_env_platform
    PlatformPackageManager().install(
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\platform.py", line 62, in install
    p.install_required_packages(force=force)
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\platform\_packages.py", line 76, in install_required_packages
    self.install_package(name, force=force)
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\platform\_packages.py", line 70, in install_package
    return self.pm.install(spec or self.get_package_spec(name), force=force)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\_install.py", line 47, in install
    pkg = self._install(spec, skip_dependencies=skip_dependencies, force=force)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\_install.py", line 130, in _install
    self.install_dependencies(pkg)
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\_install.py", line 143, in install_dependencies
    self.install_dependency(dependency)
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\manager\_install.py", line 155, in install_dependency
    dependency_compatibility = PackageCompatibility.from_dependency(dependency)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\SysGCC\esp32-master\python_env\idf5.2_py3.11_env\Lib\site-packages\platformio\package\meta.py", line 79, in from_dependency
    assert isinstance(dependency, dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
  `python -m pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
  https://docs.platformio.org/page/faq/index.html

* Report this problem to the developers
  https://github.com/platformio/platformio-core/issues

============================================================

I also left a comment in https://github.com/platformio/platform-espressif32/issues/1398#issuecomment-2198516192 that has a similar topic, noting that this platformio.ini file in the same project directory compiles successfully:

[env:esp32dev]
platform = espressif32@6.3.0
board = esp32dev
framework = espidf
monitor_speed = 115200

So clearly the problem seems to be with the install using this line:

platform_packages = 
    framework-espidf @ file:///SysGCC/esp32/esp-idf/v5.2

My objective here is to leverage some of the latest features I am working on for wolfSSL in the esp-tls layer as noted in https://github.com/espressif/esp-idf/issues/13966 - specifically those that likely won't be in a published ESP-IDF release in the immediate future.

gojimmypi commented 2 days ago

oh, and after posting my prior comment, I see in https://github.com/platformio/platform-espressif32/issues/1398#issuecomment-2198539311 from @Jason2866:

It is not possible to use the espressif IDF repo as source nor a file or directory

Are there no alternatives to installing a specific ESP-IDF version in PlatformIO?

I was looking at the documentation that had this example for the arduino platform for something similar:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
platform_packages =
    platformio/framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git

@Jason2866: Perhaps if I create a release on my branch, I can do something like this from @johnboiles?

framework-espidf @ https://github.com/tasmota/esp-idf/releases/download/v4.4.2/esp-idf-v4.4.2.zip

as noted in https://github.com/platformio/platform-espressif32/issues/889#issuecomment-1226031988 ?

Jason2866 commented 2 days ago

Yes, and I am the guy responsible for Tasmota related to all framework stuff. So the link you show is work from me ;-) If you fork ping me from there and i will help you to get a release done without that you have to reinvent the wheel

platformio / platformio-core