Closed ul-gh closed 3 years ago
Hi @ul-gh,
Thanks for the very clear report. I cloned your project master branch from GitHub and using the current ESP-IDF master branch commit I was able to reproduce different linker errors, so something is clearly not quite right!
I think probably the underlying issue is something with transitive dependencies, or at least it looks like it (i.e. some break in the chain of component A depends on component B depends on component C). Or possibly a dependency cycle where the linker command line doesn't end up with the libraries in the right place.
Can I ask exactly which ESP-IDF version you're using? I'll see if I can reproduce the exact error you're seeing by using it.
Also, if you still have the file build/esp_ajax_if.map
that matches the logs posted above, could you please attach that?
Hi @projectgus!
My current esp-idf version is v4.3-beta3, e9cf9e2978d0187b0f83a9978391217e39946517 (with added patch of PR6578)
I also tried with master branch head a few days ago, same result.
arduino-esp32 is current master with following added dependencies to make it compile:
ulrich@uhp:~/mysrc/esp32/esp_ajax_if/components/arduino-esp32$ git diff upstream
diff --git a/CMakeLists.txt b/CMakeLists.txt
index f6332f54..11b49f24 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -160,8 +160,9 @@ set(includedirs
set(srcs ${CORE_SRCS} ${LIBRARY_SRCS} ${BLE_SRCS})
set(priv_includes cores/esp32/libb64)
-set(requires spi_flash mbedtls mdns esp_adc_cal)
-set(priv_requires fatfs nvs_flash app_update spiffs bootloader_support openssl bt arduino_tinyusb main)
+set(rmaker_requires button esp_rainmaker esp_schedule json_generator json_parser qrcode rmaker_common ws2812_led)
+set(requires spi_flash mbedtls mdns esp_adc_cal wifi_provisioning esp_ipc spiffs ${rmaker_requires})
+set(priv_requires fatfs nvs_flash app_update spiffs bootloader_support openssl bt main)
if(NOT CONFIG_ARDUINO_SELECTIVE_COMPILATION OR CONFIG_ARDUINO_SELECTIVE_ArduinoOTA)
list(APPEND priv_requires esp_https_ota)
At last! This now compiles, and indeed (thanks @projectgus):
It seems there is some cyclic dependency in my components/ESPAsyncWebServer.
I have not yet found out where exactly, but setting the CMake "LINK_INTERFACE_MULTIPLICITY" which defaults to 2 to a value of 3 in that component makes the compilation succeed. (See: GNU LD linker and CMake Link order documentation)
While I am definitely a first-time-user of CMake: This seems to be a non-obvious but very likely issue one might encounter when building with the ESP-IDF build system.
IMO ESP-IDF should not choke on a wrong, automatically generated, link order. There should be a note in ESP-IDF build system documentation..
Set LINK_INTERFACE_MULTIPLICITY in components/ESPAsyncWebServer/CMakeLists.txt :
set(requires
"AsyncTCP"
)
# register_component() is now deprecated and will be removed in IDF v.5
# See: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-guides/build-system.html#no-longer-available-in-cmake
set(sources
"src/AsyncEventSource.cpp"
"src/AsyncWebSocket.cpp"
"src/SPIFFSEditor.cpp"
"src/WebAuthentication.cpp"
"src/WebHandlers.cpp"
"src/WebRequest.cpp"
"src/WebResponses.cpp"
"src/WebServer.cpp"
)
idf_component_register(
SRCS "${sources}"
INCLUDE_DIRS "src"
REQUIRES "${requires}"
)
target_compile_definitions(${COMPONENT_TARGET} PUBLIC -DESP32)
target_compile_options(${COMPONENT_TARGET} PRIVATE -fno-rtti)
# Somewhere there seems to be a cyclic dependency (arduino-esp32?) producing linker errors
# without this setting. When dependencies are fixed, following can be omitted.
set_target_properties(${COMPONENT_TARGET} PROPERTIES LINK_INTERFACE_MULTIPLICITY 3)
Hi @ul-gh,
Well done figuring this out! After a bit of further analysis, I can confirm there is a circular dependency in arduino-esp32. It depends on "main" component, where "main" is the top-level component that depends on all other components. This creates the circular dependency arduino-esp32 -> main -> arduino-esp32, which can cause linker issues with other components that also depend on arduino-esp32 (or their dependencies, in this case it was causing issues because main -> ESPAsyncWebServer -> AsyncTCP -> arduino-esp32 -> main.)
Adding LINK_INTERFACE_MULTIPLICITY
to ESPAsyncWebServer works around the issue but at the cost of a slower linker pass.
I'll submit a PR to arduino-esp32 repo to remove the "main" dependency as I don't think it's needed. If it turns out that it is needed for some reason then we can probably add LINK_INTERFACE_MULTIPLICITY
in arduino-esp32 to work around the cycles.
IMO ESP-IDF should not choke on a wrong, automatically generated, link order. There should be a note in ESP-IDF build system documentation..
Agree on both counts. Will add a note about this kind of error, possible workarounds, and some ways to find dependency cycles (as this is also not very simple!) We'll see if we can improve ESP-IDF tooling for this as well.
remove the "main" dependency as I don't think it's needed
Oh! Of course it's needed, for the case where "Autostart Arduino setup and loop on boot" is enabled. In this case, app_main is in arduino-esp32 and it needs to be able to depend on "main" where setup() and loop() functions are found...
I think we can still find a better fix for it, either with linker tricks or by making a separate libarduino_autostart.a
that only contains main.cpp
so nothing needs to depend on it...
wouldnt it be more ESPish to write it like
# Somewhere there seems to be a cyclic dependency producing linker errors
# without this setting. When dependencies are fixed, following can be omitted.
set_property(TARGET ${COMPONENT_TARGET} APPEND PROPERTY LINK_INTERFACE_MULTIPLICITY 3)
inside ESP-IDF i find it as
set_property(TARGET ${COMPONENT_LIB} APPEND PROPERTY LINK_INTERFACE_MULTIPLICITY 3)
I've been hitting this same issue for days many thanks for this! And in hindsight that loop was something I noticed and should have picked up on 😁 For the benefit of anyone using Github's global search (which is how I found this) I was getting linker errors with
base64_encode_block
(libb64
) fs::File
(File.h
)WifiServer
(WifiServer.h
)mbedtls
importsBut like was noted in the thread, this probably ends up erroring out at a non-deterministic point.
Slightly different note but related to the original comment (and what really sent me down the wrong path) is I believe ESPAsyncWebServer has a header access issue when built in esp-idf
, not sure if you saw this too @ul-gh and think it might be an issue
It's relying on the Arduino core's libb64
implementation, but that's not a part of the public interface of arduino-esp32. As a result it's listed under PRIV_INCLUDE_DIRS
, which according to ESP-IDF is only supposed to be accessible from the current component (arduino-esp32
in this case):
directory paths, must be relative to the component directory, which will be added to the include search path for this component’s source files only
I have a hunch that if the circular reference is fixed this might be an issue, but I haven't had a chance to test
Glad the workaround is helping. I've pushed the PR linked above to arduino-esp32, this should remove the need for the workaround at all.
Have a pending change also to add LINK_INTERFACE_MULTIPLICITY to the ESP-IDF build system docs.
This issue I found during the transition from PlatformIO IDE to a pure ESP-IDF setup using multiple components, i.e. arduino-esp32, ESPAsyncWebServer, among others.
After two days of trying, I can't seem to compile using IDF v4x any more. The complete project code you find here:
ESP-LiveControl
With the current ESP-IDF build system, the compilation of all source files first /succeeds/ (after some fixing of arduino-esp32, see arduino-esp32 issue 5064 but then at the final stage, I get multiple ld linker errors:
(Full build log output attached as a text file):
This is although the particular references were definitely successfully built before, as from the same build log output. A full error output I am attaching as a file.
I closely followed the project setup instructions from ESP-IDF build system documentation.
This is the project CMakeLists.txt:
This is the main component CMakeLists.txt:
This is a component CMakeLists.txt:
Am I missing anything or is the build system broken?
Regards, Ulrich Lukas build_log.txt build_log_full.txt.gz