Nested variables in platform.txt do not resolve

[me-no-dev]

Describe the bug I am adding support for ESP32S2 to ESP32 Arduino and in order to include the proper flags/libs etc, platform.txt has some variables, like compiler.path={runtime.tools.xtensa-{build.mcu}-elf-gcc.path}/bin/ that resolve fine in Arduino Builder, but seem to result in just /bin/ in sloeber.

To Reproduce Steps to reproduce the behavior:

1. Follow the instructions on how to install the development environment on your particular OS here: https://github.com/espressif/arduino-esp32/tree/esp32s2
2. Make sure that you clone the esp32s2 branch.
3. Compile an empty sketch

Expected behavior Sketch should compile fine, in fact I replaced all those nested {build.mcu} with esp32 and esp32s2 and it worked ok.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: MacOS 10.15.4
• Browser does not matter
• Version 4.3.1

Additional context https://github.com/espressif/arduino-esp32/blob/esp32s2/platform.txt

[jantje]

The instructions you point to have 2 options.

Using Arduino IDE Boards Manager (preferred) Using Arduino IDE with the development repository

Given the rest of the description I assume you are using the not preferred option. As you are using V4.3.1 I assume you do not have a platform installed that installed a tool that resolves to {runtime.tools.xtensa-{build.mcu}-elf-gcc.path} whatever {build.mcu} resolves to. To verify this assumption you can go to project->properties->C/C++ build->environment. there check what A.BUILD.MCU resolves to. Suppose that is AVR. Check what A.RUNTIME.TOOLS.XTENSA-AVR-ELF-GCC-PATH resolves to. I assume this environment variable does not exists.

[me-no-dev]

everything is there :) This is not AVR, it's ESP32. A.BUILD.MCU is also defined. Overall everything is there, but the double curlies do not resolve.

[me-no-dev]

On the preferred, non-preferred option on installing the core. I am the core maintainer, I work in the non-preferred way. This is not an issue though, I've made enough Arduino ports in my life to know my way around. The issue is only what I posted it to be.

[me-no-dev]

here are the ones that do not resolve:

[jantje]

I tried to reproduce this here and I confirm (much to my surprise) that it does not work. As the environment variable handling is internal CDT/eclipse stuff there are 2 possible root causes. 1)This is a cdt/eclipse limitation =>this will not be fixed soon(ever?) 2)Sloeber code doesn't use the CDT/Eclipse API properly=>Could be fixed easily

I fear this is number 1.

[me-no-dev]

number 1 would mean end of esp32 support in sloeber :( I currently have no other way to switch the platforms, except splitting them in two, which means 100% code duplication.

[jantje]

It works on my system when I do not expand the environment variables in the makefile. To do so: goto project properties->C/C+ build Uncheck "Expand env. Variable Refs in Makefiles" This way the double expansion is done by the environment and not by CDT/Eclipse

[me-no-dev]

I tried your suggestion, but nothing changed? i reloaded eclipse and all.

[jantje]

Check the file [project]/release/SubDir.mk If "Expand env. Variable Refs in Makefiles" is off you should see something like

sloeber.ino.cpp.o: ..\sloeber.ino.cpp
    @echo 'Building file: $<'
    @echo 'Start de C++ bouw'
    ${A.RECIPE.CPP.O.PATTERN.1}....." -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@)" -D__IN_ECLIPSE__=1 -x c++ "$<" ${A.RECIPE.CPP.O.PATTERN.2} -o "$@" ${A.RECIPE.CPP.O.PATTERN.3} ${JANTJE.EXTRA.COMPILE} ${JANTJE.EXTRA.CPP.COMPILE} ${JANTJE.EXTRA.ALL}

    @echo 'Finished building: $<'
    @echo ' '

If ${A.RECIPE.CPP.O.PATTERN.1} is not there delete the release folder and build again. If it still is not there check the "Expand env. Variable Refs in Makefiles" setting.

If it is there and it still doesn't work it may be that your os does not support nested variables.

[jantje]

@me-no-dev how is it going?

[me-no-dev]

sorry had some deadlines. the pattern is there, but variables do not expand. I tried deleting the release folder as you proposed, that changed nothing for either chip. So far my only option is to change the platform.txt when I switch chips and to and from sloeber.

[me-no-dev]

BTW I tried downloading 4.3.3 (now I remember why I am on 4.3.1) but after extracting, Mac told me that package is damaged and can not run.

[jantje]

Mac security is a pain. This is what it says above the download link

Sloeber needs java 8! Other java versions will not work!

Due to mac new security approach a simple download and extract will not work.

Follow following steps to install.

1)download with mac browser.

2)unpacked the sloeber ide zip file with the macOS native Archive Utility. Not in the download folder!!!

3)in the folder where you extracted Sloeber run: sudo xattr -r -d com.apple.quarantine Sloeber.app

Read the info on this site at: install->start here-> MacOS comments If you have download issues try another browser. Firefox seems to work fine on all oses. chrome seems to have issues.

[jantje]

sorry had some deadlines.

No problem

the pattern is there, but variables do not expand.

Could it be that mac os doesn't support Nested variables? Can you test on the command line?

[me-no-dev]

test on command line how? I am building the git master currently. Will use that to be sure it's all latest. Arduino-Builder expands the variables normally.

[jantje]

test on command line how?

I mean the mac variant of these windows commands

C:\Users\jan>set a=cpp
C:\Users\jan>set b=a%a%s
C:\Users\jan>echo %b%
acpps
C:\Users\jan>

Because ${A.RECIPE.CPP.O.PATTERN.1} is interpreted by the os ... och no ... It must be make that does the interpretation. not the os as windows needs %A.RECIPE.CPP.O.PATTERN.1%.

[jantje]

I think I have found the root cause. Sloeber internally works with uppercase variable names. Making the keys uppercase seems to fail resulting in bad names {compiler.{recipe.c}.cmd} in the platform.txt becomes ${A.COMPILER.${A.RECIPE.C}.cmd} When I put {compiler.{recipe.c}.CMD} in the platform.txt it works. Need to think on how to fix this.

[me-no-dev]

Nice! At least there is some progress! Any other "workaround" that I came up with as proposal was really stupid... so I'll spare you from them. Unfortunately not everything after such nested variables is part of another variable. Sometimes the substitution is inside a text (like toolchain path/prefix) and sometimes it's part of another variable (like above). There is no clear way to distinct one from the other. Only possibility I see is to search for variable that will resolve capitalised or not.

[jantje]

When do you expect this to be available in the boardsmanager?

[me-no-dev]

given that I am a bit pushed by superiors to release soon because it adds support for our newest chip, I would say rather sooner than later. Maybe a month, maybe less.

[jantje]

@me-no-dev can you test whether the problem is solved with the latest nightly?

[me-no-dev]

same thing. I noticed that you copy/generate platform.sloeber.txt. Seems to be exact copy.

[jantje]

same thing.

damn. Do you have a link to a platform.txt boards.txt that causes the problem?

I noticed that you copy/generate platform.sloeber.txt. Seems to be exact copy.

Yes it is generated but if no workaround is needed it is copy. I named it this way so it is clear the version control should simply ignore it.

[jantje]

same thing

have you tried opening the project properties->arduino and selected apply and close? That is what triggers the code

[me-no-dev]

platform and board txt are in the core repo branch here: https://github.com/espressif/arduino-esp32/tree/esp32s2 I did try restart, delete the release folder, change boards and everything that came to mind. With and without that C++ build option you mentioned before (something about expanding the variables). I noticed the generated file because I had already a file with that name in my local copy (as well as platform.arduino.txt) to make it easy for me to switch between sloeber and the IDE.

[jantje]

I tried to build the default sketch based on the repository you provided and the first error I get is not related to nested variables

13:50:03 **** Build of configuration Release for project issue1184 ****
"C:\\sloeber\\arduinoPlugin\\tools\\make\\make" all 
C:/sloeber/arduinoPlugin/tools/make/make --no-print-directory pre-build
cmd /c if exist "C:\workspaces\embedded\issue1184\partitions.csv" copy /y "C:\workspaces\embedded\issue1184\partitions.csv" "C:\workspaces\embedded\issue1184\Release\partitions.csv"
cmd /c if not exist "C:\workspaces\embedded\issue1184\Release\partitions.csv" copy "C:/Users/jan/git/arduino-esp32\tools\partitionsdefault.csv" "C:\workspaces\embedded\issue1184\Release\partitions.csv"
The syntax of the command is incorrect.
make[1]: *** [makefile:104: pre-build] Error 1
make: *** [makefile:64: all] Error 2
"C:/sloeber/arduinoPlugin/tools/make/make all" terminated with exit code 2. Build might be incomplete.

13:50:04 Build Failed. 2 errors, 0 warnings. (took 1s.195ms)

These prebuild commands are very similar to what I have seen in the discussion around STM32 in issue #1143. Start from here https://github.com/Sloeber/arduino-eclipse-plugin/issues/1143#issuecomment-588041278 There we also noticed that although the commands are perfectly legit ; make doesn't like them.

I personally do not like the commands because they are specific for Arduino IDE behaviour; where the arduino framework is intended to be tool agnostic. Arduino behaviour I'm thinking of is 1) Arduino ide does not build the code at the sketch location 2) arduino ide does not copy *.cvs files over to the build location

Given your boards.txt and platform.txt. What I think would be way better is in boards.txt

esp32s2.menu.PartitionScheme.default=Default 4MB with spiffs (1.2MB APP/1.5MB SPIFFS)
esp32s2.menu.PartitionScheme.default.build.partitions={runtime.platform.path}/tools/partitions/default.csv
esp32s2.menu.PartitionScheme.custom=partitions.csv file provided in the sketch
esp32s2.menu.PartitionScheme.custom.build.partitions={build.source.path}\partitions.csv

in the platform.txt delete the custom build commands and change this line

recipe.objcopy.eep.pattern={tools.gen_esp32part.cmd} -q "{build.partitions}" "{build.path}/{build.project_name}.partitions.bin"

I did these changes and then I hit the nested variables issue. The problem is with

build.mcu=esp32s2
compiler.path={runtime.tools.xtensa-{build.mcu}-elf-gcc.path}/bin/

Sloeber does the capitalisation just fine but, esp32s2 is not a "variable value" and not a "variable name" and as such not capitalized and therefore Sloeber gets COMPILER.PATH={RUNTIME.TOOLS.WTENSA-esp32s2-ELF-GCC.PATH}/bin/ and that should be COMPILER.PATH={RUNTIME.TOOLS.WTENSA-ESP32S2-ELF-GCC.PATH}/bin/ (One should expect this to work in case insensitive windows,...but it doesn't)

This is something I will need to think about how I can fix it.

Anyway I hacked the platform.txt to keep going (because the above proposal is not yet tested) First I needed to run the tools/get.exe And now it builds but fails with

15:32:07 **** Build of configuration Release for project issue1184 ****
"C:\\sloeber\\arduinoPlugin\\tools\\make\\make" all 
'Building file: ..\sloeber.ino.cpp'
'Start de C++ bouw'
"C:/Users/jan/git/arduino-esp32/tools/xtensa-esp32s2-elf/bin/xtensa-esp32s2-elf-g++"   -DF_CPU=160000000L -DARDUINO=10812 -DARDUINO_ESP32S2_DEV -DARDUINO_ARCH_ARDUINO-ESP32 "-DARDUINO_BOARD=\"ESP32S2_DEV\"" -DARDUINO_VARIANT="esp32s2"  -DESP32 -DCORE_DEBUG_LEVEL=5 -DBOARD_HAS_PSRAM   -I"C:\Users\jan\git\arduino-esp32\cores\esp32" -I"C:\Users\jan\git\arduino-esp32\variants\esp32s2" -MMD -MP -MF"sloeber.ino.cpp.d" -MT"sloeber.ino.cpp.o" -D__IN_ECLIPSE__=1 -x c++ "..\sloeber.ino.cpp"   -o "sloeber.ino.cpp.o"
<command-line>: warning: ISO C++11 requires whitespace after the macro name
In file included from ..\sloeber.ino.cpp:7:
C:\Users\jan\git\arduino-esp32\cores\esp32/Arduino.h:32:10: fatal error: freertos/FreeRTOS.h: No such file or directory
 #include "freertos/FreeRTOS.h"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [subdir.mk:20: sloeber.ino.cpp.o] Error 1
"C:/sloeber/arduinoPlugin/tools/make/make all" terminated with exit code 2. Build might be incomplete.

15:32:07 Build Failed. 2 errors, 0 warnings. (took 485ms)

I'm unsure on how to continue.

[me-no-dev]

hahahaha I'm actually glad that you hit this error :) Sloeber automatically attaches the AVR FreeRTOS lib if it sees this include. Which is wrong. You need to go to the Add libs menu and uncheck FreeRTOS

[me-no-dev]

oh no, you are missing all includes. The command did not expand properly.

[me-no-dev]

Here is a correct command for esp32s2 (from sloeber with modified platform.txt)

"/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/xtensa-esp32s2-elf/bin/xtensa-esp32s2-elf-g++" -DHAVE_CONFIG_H -DMBEDTLS_CONFIG_FILE="mbedtls/esp_config.h" -DUNITY_INCLUDE_CONFIG_H -DWITH_POSIX -D_GNU_SOURCE -DIDF_VER="v4.2-dev-1575-ga3520970f" -DESP_PLATFORM   "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/config" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/newlib/platform_include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/freertos/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/freertos/xtensa/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/heap/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/log/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/lwip/include/apps" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/lwip/include/apps/sntp" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/lwip/lwip/src/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/lwip/port/esp32/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/lwip/port/esp32/include/arch" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/soc/src/esp32s2" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/soc/src/esp32s2/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/soc/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_rom/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_common/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_system/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/xtensa/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/xtensa/esp32s2/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp32s2/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/driver/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/driver/esp32s2/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_ringbuf/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/efuse/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/efuse/esp32s2/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/espcoredump/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_timer/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/soc/soc/esp32s2" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/soc/soc/esp32s2/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/soc/soc/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/vfs/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_wifi/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_wifi/esp32s2/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_event/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_netif/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_eth/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/tcpip_adapter/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/app_trace/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/mbedtls/port/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/mbedtls/mbedtls/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/mbedtls/esp_crt_bundle/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/bootloader_support/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/app_update/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/spi_flash/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/wpa_supplicant/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/wpa_supplicant/port/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/wpa_supplicant/include/esp_supplicant" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/nvs_flash/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/pthread/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/asio/asio/asio/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/asio/port/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/cbor/port/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/coap/port/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/coap/port/include/coap" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/coap/libcoap/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/coap/libcoap/include/coap2" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/console" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/nghttp/port/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/nghttp/nghttp2/lib/includes" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp-tls" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_gdbstub/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_hid/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/tcp_transport/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_http_client/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_http_server/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_https_ota/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/protobuf-c/protobuf-c" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/protocomm/include/common" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/protocomm/include/security" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/protocomm/include/transports" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/mdns/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_local_ctrl/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/sdmmc/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_serial_slave_link/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/esp_websocket_client/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/expat/expat/expat/lib" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/expat/port/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/wear_levelling/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/fatfs/diskio" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/fatfs/vfs" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/fatfs/src" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/freemodbus/common/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/idf_test/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/idf_test/include/esp32s2" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/jsmn/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/json/cJSON" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/libsodium/libsodium/src/libsodium/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/libsodium/port_include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/mqtt/esp-mqtt/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/openssl/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/perfmon/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/spiffs/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/ulp/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/unity/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/unity/unity/src" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/wifi_provisioning/include" "-I/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/tools/sdk/esp32s2/include/fb_gfx/include" -mlongcalls -ffunction-sections -fdata-sections -fstrict-volatile-bitfields -Wno-error=unused-function -Wno-error=unused-but-set-variable -Wno-error=unused-variable -Wno-error=deprecated-declarations -Wno-unused-parameter -Wno-sign-compare -ggdb -Og -fstack-protector -std=gnu++11 -fexceptions -fno-rtti  -MMD -c -DF_CPU=240000000L -DARDUINO=10812 -DARDUINO_ESP32S2_DEV -DARDUINO_ARCH_ESP32 '-DARDUINO_BOARD="ESP32S2_DEV"' -DARDUINO_VARIANT="esp32s2"  -DESP32 -DCORE_DEBUG_LEVEL=4 -DBOARD_HAS_PSRAM   -I"/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/cores/esp32" -I"/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/variants/esp32s2" -I"/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/libraries/SPI/src" -I"/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/libraries/Wire/src" -I"/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/libraries/WiFi/src" -I"/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/libraries/SPI" -I"/Users/ficeto/Documents/Arduino/hardware/espressif/esp32/libraries/Wire" -MMD -MP -MF"st7920.cpp.d" -MT"st7920.cpp.o" -D__IN_ECLIPSE__=1 -x c++ "../st7920.cpp"   -o "st7920.cpp.o"

[me-no-dev]

On the commands for partitions. Their purpose is not just to provide a way for the user to include his own partitions, but also such to be provided with particular examples, which will fail if the user does not select the proper menu option (needs to know to do so also). Kinda not the intended result. Another incompatibility between Arduino and Sloeber is -DMBEDTLS_CONFIG_FILE="mbedtls/esp_config.h", which works in Arduino but not in Sloeber (It is included in a header as #include MBEDTLS_CONFIG_FILE) and this is not an Arduino header, so nothing that I can do

[jantje]

I see you making a couple statements that do not correspond with my perception. I propose to handle them one by one. Or we'll get confused and mix up things.

I think it is best to first tackle the nested varaible esp32 version compiles in one way or another in Sloeber. I ran in some troubles again and restarted. In platform.sloeber.txt I replaced all build.mcu with build.mcu.env and I added build.mcu.env=ESP32S2

And now I get a linker error

'Start de samenvoeger'
"C:/Users/jan/git/arduino-esp32/tools/xtensa-esp32s2-elf/bin/xtensa-ESP32S2-elf-g++" "-Wl,--Map=C:\workspaces\embedded\issue1184\Release/issue1184.map" "-LC:/Users/jan/git/arduino-esp32/tools/sdk/ESP32S2/lib" "-LC:/Users/jan/git/arduino-esp32/tools/sdk/ESP32S2/ld" -mlongcalls -T esp32s2.rom.ld -T esp32s2.rom.libgcc.ld -T esp32s2.rom.newlib-data.ld -T esp32s2.rom.newlib-funcs.ld -T esp32s2.rom.spiflash.ld -T esp32s2_out.ld -T esp32s2.project.ld -T esp32s2.peripherals.ld  -Wl,--cref -fno-rtti -fno-lto -u esp_app_desc -u vfs_include_syscalls_impl -u pthread_include_pthread_impl -u pthread_include_pthread_cond_impl -u pthread_include_pthread_local_storage_impl -u app_main -u call_user_start_cpu0 -u ld_include_panic_highint_hdl -Wl,--gc-sections -Wl,--undefined=uxTopUsedPriority -u newlib_include_locks_impl -u newlib_include_heap_impl -u newlib_include_syscalls_impl -u newlib_include_pthread_impl -u __cxa_guard_dummy  -Wl,--start-group    .\sloeber.ino.cpp.o    C:\workspaces\embedded\issue1184\Release/arduino.ar  -lxtensa -lmbedtls -lefuse -lbootloader_support -lapp_update -lspi_flash -lesp_system -lsoc -lvfs -lesp_eth -ltcpip_adapter -lesp_netif -lesp_event -lwpa_supplicant -lnvs_flash -lesp_wifi -llwip -llog -lheap -lesp_ringbuf -ldriver -lpthread -lespcoredump -lesp32s2 -lesp_common -lesp_timer -lfreertos -lnewlib -lcxx -lapp_trace -lasio -lcbor -lcoap -lconsole -lnghttp -lesp-tls -lesp_gdbstub -lesp_hid -ltcp_transport -lesp_http_client -lesp_http_server -lesp_https_ota -lprotobuf-c -lprotocomm -lmdns -lesp_local_ctrl -lsdmmc -lesp_serial_slave_link -lesp_websocket_client -lexpat -lwear_levelling -lfatfs -lfreemodbus -ljsmn -ljson -llibsodium -lmqtt -lopenssl -lperfmon -lspiffs -lulp -lunity -lwifi_provisioning -lfb_gfx -lasio -lcbor -lcoap -lesp_gdbstub -lesp_hid -lesp_https_ota -lesp_local_ctrl -lesp_serial_slave_link -lesp_websocket_client -lexpat -lfreemodbus -ljsmn -llibsodium -lmqtt -lperfmon -lunity -lwifi_provisioning -lprotocomm -lprotobuf-c -ljson -lfb_gfx -lmdns -lconsole -lfatfs -lsdmmc -lwear_levelling -lopenssl -lspiffs -lxtensa -lmbedtls -lefuse -lbootloader_support -lapp_update -lspi_flash -lesp_system -lsoc -lvfs -lesp_eth -ltcpip_adapter -lesp_netif -lesp_event -lwpa_supplicant -lnvs_flash -lesp_wifi -llwip -llog -lheap -lesp_ringbuf -ldriver -lpthread -lespcoredump -lesp32s2 -lesp_common -lesp_timer -lfreertos -lnewlib -lcxx -lapp_trace -lnghttp -lesp-tls -ltcp_transport -lesp_http_client -lesp_http_server -lulp -lmbedtls -lmbedcrypto -lmbedx509 -lsoc_esp32s2 -lcoexist -lcore -lespnow -lmesh -lnet80211 -lpp -lrtc -lsmartconfig -lphy -lxtensa -lmbedtls -lefuse -lbootloader_support -lapp_update -lspi_flash -lesp_system -lsoc -lvfs -lesp_eth -ltcpip_adapter -lesp_netif -lesp_event -lwpa_supplicant -lnvs_flash -lesp_wifi -llwip -llog -lheap -lesp_ringbuf -ldriver -lpthread -lespcoredump -lesp32s2 -lesp_common -lesp_timer -lfreertos -lnewlib -lcxx -lapp_trace -lnghttp -lesp-tls -ltcp_transport -lesp_http_client -lesp_http_server -lulp -lmbedtls -lmbedcrypto -lmbedx509 -lsoc_esp32s2 -lcoexist -lcore -lespnow -lmesh -lnet80211 -lpp -lrtc -lsmartconfig -lphy -lxtensa -lmbedtls -lefuse -lbootloader_support -lapp_update -lspi_flash -lesp_system -lsoc -lvfs -lesp_eth -ltcpip_adapter -lesp_netif -lesp_event -lwpa_supplicant -lnvs_flash -lesp_wifi -llwip -llog -lheap -lesp_ringbuf -ldriver -lpthread -lespcoredump -lesp32s2 -lesp_common -lesp_timer -lfreertos -lnewlib -lcxx -lapp_trace -lnghttp -lesp-tls -ltcp_transport -lesp_http_client -lesp_http_server -lulp -lmbedtls -lmbedcrypto -lmbedx509 -lsoc_esp32s2 -lcoexist -lcore -lespnow -lmesh -lnet80211 -lpp -lrtc -lsmartconfig -lphy -lxtensa -lmbedtls -lefuse -lbootloader_support -lapp_update -lspi_flash -lesp_system -lsoc -lvfs -lesp_eth -ltcpip_adapter -lesp_netif -lesp_event -lwpa_supplicant -lnvs_flash -lesp_wifi -llwip -llog -lheap -lesp_ringbuf -ldriver -lpthread -lespcoredump -lesp32s2 -lesp_common -lesp_timer -lfreertos -lnewlib -lcxx -lapp_trace -lnghttp -lesp-tls -ltcp_transport -lesp_http_client -lesp_http_server -lulp -lmbedtls -lmbedcrypto -lmbedx509 -lsoc_esp32s2 -lcoexist -lcore -lespnow -lmesh -lnet80211 -lpp -lrtc -lsmartconfig -lphy -lhal -lm -lnewlib -lgcc -lstdc++ -lpthread -lapp_trace -lgcov -lapp_trace -lgcov -lc -Wl,--end-group -Wl,-EL -o "C:\workspaces\embedded\issue1184\Release/issue1184.elf"
c:/users/jan/git/arduino-esp32/tools/xtensa-esp32s2-elf/bin/../lib/gcc/xtensa-esp32s2-elf/8.2.0/../../../../xtensa-esp32s2-elf/bin/ld.exe: C:/Users/jan/git/arduino-esp32/tools/sdk/ESP32S2/lib\librtc.a(rtc.o)(.text.rtc_pad_gpio_wakeup+0x50b7fb000000a9): could not decode instruction; possible configuration mismatch
'Finished building: issue1184.elf'

' '

[me-no-dev]

this error is normal (bug still present upstream). you have successfully compiled the firmware :)

[jantje]

ok good. The next step fails. I'll take a look at that now.

[jantje]

The next failings were due to the global find and replace of the build.mcu to build.mcu.env

So on behalf of the build of the nested variables I think there are 2 issues with ESP32 and Sloeber 1) The nested variables combined with uppercasing in Sloeber (up to me to fix) 2) The prebuild commands

Other things you mention 1)Sloeber automatically attaches the AVR FreeRTOS lib if it sees this include. Which is wrong. You need to go to the Add libs menu and uncheck FreeRTOS I think this is #1163. Which is high on my todo list. It is easier to just delete the FreeRTOS folder under libraries. 2)-DMBEDTLS_CONFIG_FILE="mbedtls/esp_config.h", This has been fixed.

So IMHO the only thing needing discussion is the prebuild commands. I do not understand why you need to copy the file. Why not directly point to the file that is copied? In other words why not replace recipe.objcopy.eep.pattern={tools.gen_esp32part.cmd} -q "{build.partitions}" "{build.path}/{build.project_name}.partitions.bin" with recipe.objcopy.eep.pattern={tools.gen_esp32part.cmd} -q "{runtime.platform.path}/tools/partitions/{build.partitions}" "{build.path}/{build.project_name}.partitions.bin" This does not support custom csv files but that could be easily hacked around using nested variables. And no more need for prebuild commands.

[jantje]

Seems the uppercasing is something CDT does in windows. I will have to find a way around that.

[me-no-dev]

i wish I could help in any way, but how Sloeber creates the makefiles is a mystery to me :)

[jantje]

i wish I could help in any way, but how Sloeber creates the makefiles is a mystery to me :)

The problem is not how the makefile is created. The problem is with how CDT (in windows) handles environment variables.

In principle the makefile generation is CDT code. CDT managed build allows to have your own makefile generator. Because of the requirement in Arduino to build the archive and the hex in the same project I copied that and adopted it. However everyone in the CDT community is convinced the current managed build is not worth investing in extending. It needs to be replaced with something better. A daunting task that will take time, but we are working on it.

PS I'm still hoping for some feedback on the prebuild commands

[me-no-dev]

as you said earlier those commands are a separate issue. You are asking me to make it not auto, I am not yet sure that I want that, so I'll be looking for another way, but not now :) code works in the IDE and in Sloeber on non-windows

[jantje]

You are asking me to make it not auto,

What do you mean with that. The only difference I see between what I propose and what the current implementation does is in the situation there is a partitions.csv. The current way the sketch will always use this file. What I propose the sketch will use what is selected in the menu.

[me-no-dev]

Yes, the point is to always use the included with the sketch partitions, if such file exists. That not to depend on menu selection (user might not be aware, so on). We have projects that depend on particular partition scheme and will not work with any other (image recognition and such take a lot of space). I know I could have added it as menu, but it just is not the point :)

[jantje]

Thanks for the info. It's basically opposite from how I like it (which is having things explicit not implicit). Anyway that is how you want it that is how it needs to be. That leaves me with 3 things to fix for esp32

1)The nested variables combined with uppercasing in Sloeber 
2)The prebuild commands (windows only)
3)#1163 

As long as 1 isn't fixed there is no real need to look at 2. For 1 I asked CDT whether I could fix the casing in CDT which is part of problem 1. This will take some time to discuss. So I'm back to 3 for now.

[me-no-dev]

I totally agree, 2 is something down the road. It can be discussed, changed and so on (hopefully the functionality can be kept) but non the less it is not critical :)

a bit of side question (just a question): I kinda remember there being an option to not regenerate the makefiles each build. I have an option to just turn them off now (which is fine once generated). Do you have an idea of what I am talking about? Makefile generation seem to take some time each click of build or upload.

[jantje]

a bit of side question (just a question): I kinda remember there being an option to not regenerate the makefiles each build. I have an option to just turn them off now (which is fine once generated). Do you have an idea of what I am talking about? Makefile generation seem to take some time each click of build or upload.

The makefile should not be recreated each build and on windows it doesn't do so. I can't say for sure on mac. However, if you use .ino files the sloeber.ino.cpp file needs to be generated. And for that the indexer needs to be up to dated. Due to the autosave the indexer is probably not up to date.

Anyway you can turn of the makefile generation in C/C++ build->generate makefiles automatically However adding/removing files or changing any project properties will no longer be reflected in the makefile. Also the sloeber.ino.cpp generation is still going to happen.

PS: in windows/linux you can see the current ongoing actions (like indexing) in the right bottom of the eclipse window.

[me-no-dev]

it's more of a: goes blank for a bit, message is "building project". I'll try not to use ino, I usually don't and always write code that compiles without Arduino's stupid declaration functionality.

[jantje]

When you click build eclipse starts the eclipse build chain. For sloeber this looks like

The first step only finishes when the indexer is done because Sloeber queries the indexer to do the "Arduino's stupid declaration functionality". Only then the CDT step is done which may or may not rebuild the makefiles.

as from eclipse point of view: this is all "building project"

[jantje]

Status (as a reminder for me) 1)The nested variables combined with uppercasing in Sloeber It seems like there is a relative easy fix in CDT. Needs more testing though and CDT acceptance 2)The prebuild commands (windows only) The prebuilds needs 2 fixes I can do in the workaround The first fix is to remove the cmd -c The second fix is to replace { with {{. The second fix should be aplied to all of the platform.txt and should be rare 3)#1163 This seems a tough nut to crack. I asked help from CDT dev but uptill now no response.

[jantje]

update 1) I did the changes for CDT and pushed them. It got refused. Did the full n solution. Part of it is accepted. I'm waiting for acceptance of the final changes 2) the first fix was already implemented 3) I didn't get input from CDT. I've been testing and found a better way to create projects but my fist test still had a library included.

As the new CDT will be September a release of sloeber with this working will not be before September

[me-no-dev]

Thanks @jantje ! September is much better than never ;)

[ESP32DE]

Status (as a reminder for me) 1)The nested variables combined with uppercasing in Sloeber It seems like there is a relative easy fix in CDT. Needs more testing though and CDT acceptance 2)The prebuild commands (windows only) The prebuilds needs 2 fixes I can do in the workaround The first fix is to remove the cmd -c The second fix is to replace { with {{. The second fix should be aplied to all of the platform.txt and should be rare 3)#1163 This seems a tough nut to crack. I asked help from CDT dev but uptill now no response.

hi jantie thanks for your effort.

FYI : i changed the (windows prebuild ) lines like this:

plattform.txt

# Check if custom partitions exist
# # rudi
# # recipe.hooks.prebuild.1.pattern=bash -c "[ ! -f {build.source.path}/partitions.csv ] || cp -f {build.source.path}/partitions.csv {build.path}/partitions.csv"
# # recipe.hooks.prebuild.2.pattern=bash -c "[ -f {build.path}/partitions.csv ] || cp {runtime.platform.path}/tools/partitions/{build.partitions}.csv {build.path}/partitions.csv"
recipe.hooks.prebuild.1.pattern.windows=@cmd /c if exist "{build.source.path}\partitions.csv" COPY /y "{build.source.path}\partitions.csv" "{build.path}\partitions.csv"
recipe.hooks.prebuild.2.pattern.windows=@cmd /c if not exist "{build.path}\partitions.csv" COPY "{runtime.platform.path}\tools\partitions\\{build.partitions}.csv" "{build.path}\partitions.csv"

the second problem was in this part

COPY "{runtime.platform.path}\tools\partitions\\{build.partitions}.csv"

i change and use double backslashes before {build.partitions}.csv

@me-no-dev the windows cmd runs now well and find the partitions file fine. hope this helps.

esp32-s2 (esp32s2f2) works.

best wishes

edit typo (partions->partitions)

[jantje]

@ESP32DE Good thinking. Most tools won't have problems with double \