ESP-IDF: Import of non-existent variable ''projenv''

[cguimaraes]

Configuration

Operating system: MacOS

PlatformIO Version (platformio --version): PlatformIO Core, version 6.0.3rc1

Description of problem

Before the update to PlatformIO Core v6.0.0, I was able to globally define additional macros (e.g. -DFoo) in a project from any extra_script.py of used libraries. Just like this:

Import('env')
from os.path import join, realpath

cppdefines = []

framework = env.get("PIOFRAMEWORK")[0]
if framework == 'zephyr':
  cppdefines=["FOO_ZEPHYR"]

elif framework == 'arduino':
  cppdefines=["FOO_ARDUINO"]

elif framework == 'espidf':
  cppdefines=["FOO_ESPIDF"]

env.Append(CPPDEFINES=cppdefines)

global_env = DefaultEnvironment()
global_env.Append(CPPDEFINES=cppdefines)

After the update to PlatformIO Core >= v6.0.0, any file in the project/src will not take them into account (confirmed by running with platformio run -v ).

It seems that adding the following lines solves the problem for a subset of frameworks.

Import('projenv')
projenv.Append(CPPDEFINES=cppdefines)

However, ESP-IDF framework still fails due to Import of non-existent variable ''projenv'' .

Steps to Reproduce

1. Create a dummy library and set extra_script_py (the one above for example).
2. Create a dummy source that uses the library.
3. Check if any of the FOO_* is being defined and observe errors

Actual Results

Processing az-delivery-devkit-v4 (platform: espressif32; board: az-delivery-devkit-v4; framework: espidf)
----------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/az-delivery-devkit-v4.html
PLATFORM: Espressif 32 (4.4.0) > AZ-Delivery ESP-32 Dev Kit C V4
HARDWARE: ESP32 240MHz, 520KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-espidf @ 3.40302.0 (4.3.2)
 - tool-cmake @ 3.16.4
 - tool-esptoolpy @ 1.30300.0 (3.3.0)
 - tool-mkfatfs @ 2.0.1
 - tool-mklittlefs @ 1.203.210628 (2.3)
 - tool-mkspiffs @ 2.230.0 (2.30)
 - tool-ninja @ 1.9.0
 - toolchain-esp32ulp @ 1.22851.191205 (2.28.51)
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3
WARNING: There was an error checking the latest version of pip.
Reading CMake configuration...

*** Import of non-existent variable ''projenv''
File "extra_script.py", line 1, in <module>

Expected Results

It to build.

If problems with PlatformIO Build System:

The content of platformio.ini:

[env:az-delivery-devkit-v4]
platform = espressif32
board = az-delivery-devkit-v4
framework = espidf

Additional info

N/A

[ivankravets]

*** Import of non-existent variable ''projenv'' File "extra_script.py", line 1, in

What is your extra script? It seems you have a broken import directive. It should be Import("projenv").

[cguimaraes]

The one working before Platformio Core >= 6.0.0: https://github.com/eclipse-zenoh/zenoh-pico/blob/master/extra_script.py

With the partial fix mentioned in this issue: https://github.com/eclipse-zenoh/zenoh-pico/pull/92/files

[cguimaraes]

Note that, for Arduino and Zephyr projects the partial fix worked without major issues, but it fails for ESP-IDF.

[cguimaraes]

@ivankravets I just noticed that you closed the issue.

It seems you have a broken import directive.

You observation is correct. Still, my question is why such directive does not exist whenever I have a project for espidf framework?

[ivankravets]

The projenv environment is only available for the POST script. Could you share a simple project to reproduce this issue?

[cguimaraes]

Here is an out-of-the-box example that allows to reproduce the issue: https://github.com/cguimaraes/fix-89-example-for-pio

For some reason, the following lines are not properly defining ZENOH_ARDUINO_ESP32 or ZENOH_ESPIDF for any file in the src directory of the main project, thus the #error "Unknown platform".

global_env = DefaultEnvironment()
global_env.Append(CPPDEFINES=cppdefines)

In file included from .pio/libdeps/az-delivery-devkit-v4/zenoh-pico/include/zenoh-pico/collections/element.h:18,
                 from .pio/libdeps/az-delivery-devkit-v4/zenoh-pico/include/zenoh-pico/collections/intmap.h:18,
                 from .pio/libdeps/az-delivery-devkit-v4/zenoh-pico/include/zenoh-pico/utils/properties.h:19,
                 from .pio/libdeps/az-delivery-devkit-v4/zenoh-pico/include/zenoh-pico/api/config.h:18,
                 from .pio/libdeps/az-delivery-devkit-v4/zenoh-pico/include/zenoh-pico.h:19,
                 from /Users/user/Documents/TestEnvironment/fix-89-example-for-pio/arduino/src/zn_pub.ino:20:
.pio/libdeps/az-delivery-devkit-v4/zenoh-pico/include/zenoh-pico/system/platform.h:30:2: error: #error "Unknown platform"
 #error "Unknown platform"

Note that, in Platformio Core <=v6.0.0 it was working just fine.

The PR mentioned in a previous post ( https://github.com/eclipse-zenoh/zenoh-pico/pull/92/files ) fixes this, but not for the ESPIDF. But according to your last message, it seems that Arduino/Zephyr builds should also fail for the same reason which is not the case.

[ivankravets]

That was the bug in PlatformIO Core 5. PlatformIO Core 6.0 fixes it and introduces 2 isolated build environments: global and project.

1. Please upgrade to the latest PlatformIO Core 6.0.3-dev using the pio upgrade --dev command
2. Try this code:

Import("projenv")

projenv.Append(CPPDEFINES=cppdefines)

Now, macros will go to the "src" files from the project.

[cguimaraes]

That bring us to the initial issue on this post whenever the target framework is espidf.

Processing az-delivery-devkit-v4 (platform: espressif32; board: az-delivery-devkit-v4; framework: espidf)
----------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/az-delivery-devkit-v4.html
PLATFORM: Espressif 32 (4.4.0) > AZ-Delivery ESP-32 Dev Kit C V4
HARDWARE: ESP32 240MHz, 520KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-espidf @ 3.40302.0 (4.3.2)
 - tool-cmake @ 3.16.4
 - tool-esptoolpy @ 1.30300.0 (3.3.0)
 - tool-mkfatfs @ 2.0.1
 - tool-mklittlefs @ 1.203.210628 (2.3)
 - tool-mkspiffs @ 2.230.0 (2.30)
 - tool-ninja @ 1.9.0
 - toolchain-esp32ulp @ 1.22851.191205 (2.28.51)
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3
WARNING: There was an error checking the latest version of pip.
Reading CMake configuration...

*** Import of non-existent variable ''projenv''

[ivankravets]

Could you re-run pio upgrade --dev. Does it work now?

[cguimaraes]

Unfortunately, the issue still persists. I have also executed pio pkg update after pio upgrade --dev.

% platformio run
Processing az-delivery-devkit-v4 (platform: espressif32; board: az-delivery-devkit-v4; framework: espidf)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/az-delivery-devkit-v4.html
PLATFORM: Espressif 32 (4.4.0) > AZ-Delivery ESP-32 Dev Kit C V4
HARDWARE: ESP32 240MHz, 520KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-espidf @ 3.40302.0 (4.3.2)
 - tool-cmake @ 3.16.4
 - tool-esptoolpy @ 1.30300.0 (3.3.0)
 - tool-ninja @ 1.9.0
 - toolchain-esp32ulp @ 1.22851.191205 (2.28.51)
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3
WARNING: There was an error checking the latest version of pip.
Reading CMake configuration...

*** Import of non-existent variable ''projenv''
File "extra_script.py", line 1, in <module>

[ivankravets]

Did you remove “pre:” prefix from extra scripts path in PlatformIO.ini?

[cguimaraes]

This is the platformio.ini of the main project:

[env:az-delivery-devkit-v4]
platform = espressif32
board = az-delivery-devkit-v4
framework = espidf
lib_deps = https://github.com/eclipse-zenoh/zenoh-pico

In Zenoh-Pico library, there is the following library.json:

{
    "name": "zenoh-pico",
    "license": "Apache-2.0",
    "build": {
        "extraScript": "extra_script.py"
    },
    "dependencies":
    {
       "BluetoothSerial": "1.0"
    }
}

[ivankravets]

I see the problem. It is linked with the ESP-IDF build script on the PlatformIO side. We will think about how to resolve this issue.

[ivankravets]

I've just fixed this issue but a new one arose. Working with projenv and ESP-IDF framework does not have any effect on project source files. Please file a new issue in this repository https://github.com/platformio/platform-espressif32/issues

Thanks!

[cguimaraes]

@ivankravets thank you for the fix. But, contrary to what you said, the build_flags are being set in the main project as expected. Meaning that it fixed all the issues I was witnessing.

[cguimaraes]

@ivankravets it seems that this issue has been introduced again for ESP-IDF. Any flag being set in cppdefines in the extra_script.py of my library is not taking effect on the main project (which makes use of the library).

[ivankravets]

@cguimaraes, sorry for the latest response. Which env do you use to expose flags to the main project? Our docs was updated, see https://docs.platformio.org/en/latest/scripting/construction_environments.html

[cguimaraes]

In extra_script.py of the library, I have the following lines to expose flags into the main project:

# pass flags to a global build environment (for all libraries, etc)
global_env = DefaultEnvironment()
global_env.Append(CPPDEFINES=CPPDEFINES)

where CPPDEFINES is

CPPDEFINES = ["ZENOH_ESPIDF"]

The issue so far is only with ESP-IDF platform. The same approach is working as expected for MbedOS, Zephyr and Arduino ports.

[ivankravets]

Please open a seprate issue at https://github.com/platformio/platform-espressif32/issues

Thanks!