Closed ammaree closed 4 years ago
ammaree
commented
4 years ago @dobairoland Any suggestions for solving this? I have not been able to trace any direct/clear reference to esptool_py_flash_target in the IDF so assume it is build up from variables or constructed in some other way?
dobairoland
commented
4 years ago Hi @ammaree. Thank for reporting a possible issue. I assume that you cannot reproduce this with the projects in the example directory. Right?
Function esptool_py_flash_target is defined here:
https://github.com/espressif/esp-idf/blob/master/components/esptool_py/project_include.cmake#L179
I'm not sure how to solve this. I'll ask my collegue.
@renzbagaporo PTAL
renzbagaporo
commented
4 years ago Hi @ammaree,
Seconding @dobairoland with suggestion to try building one of the examples. Also posting the bad commit (the one you updated to) and the good commit (previous working version) will also be helpful in debugging this issue.
ammaree
commented
4 years ago @dobairoland @renzbagaporo
Have tried to build real_time_stats example for master and it succeeds. The commit ID's I do not have have an not sure how to get at this stage. As mentioned in the first posting the last working update was as of ~27th April and built successfully up to around midnight on 30th April It stopped working after updating on 1 May so one of the commits in the period between those days caused the problem.
ammaree
commented
4 years ago Just checked, cannot even run build.py menuconfig against my project, failed with exact same error
Executing action: menuconfig
Running cmake in directory c:\dropbox\devs\ws\z-appl\irmacs\build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 --warn-uninitialized -DIDF_TARGET=esp32 -DCCACHE_ENABLE=1 c:\dropbox\devs\ws\z-appl\irmacs"...
Warn about uninitialized values.
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.26.1.windows.1")
-- ccache will be used for faster recompilation
-- The C compiler identification is GNU 8.2.0
-- The CXX compiler identification is GNU 8.2.0
-- The ASM compiler identification is GNU
-- Found assembler: C:/Users/andre/.espressif/tools/xtensa-esp32-elf/esp-2020r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe
-- Check for working C compiler: C:/Users/andre/.espressif/tools/xtensa-esp32-elf/esp-2020r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe
-- Check for working C compiler: C:/Users/andre/.espressif/tools/xtensa-esp32-elf/esp-2020r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Users/andre/.espressif/tools/xtensa-esp32-elf/esp-2020r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++.exe
-- Check for working CXX compiler: C:/Users/andre/.espressif/tools/xtensa-esp32-elf/esp-2020r1-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building ESP-IDF components for target esp32
-- Found PythonInterp: C:/Users/andre/.espressif/python_env/idf4.2_py3.8_env/Scripts/python.exe (found version "3.8.2")
-- Could NOT find Perl (missing: PERL_EXECUTABLE)
CMake Error at C:/Dropbox/devs/ws/z-sdk/esp-idf/components/partition_table/CMakeLists.txt:93 (esptool_py_flash_target):
Unknown CMake command "esptool_py_flash_target".
-- Configuring incomplete, errors occurred!
See also "C:/Dropbox/devs/ws/z-appl/irmacs/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1@ammaree I suppose you don't have that project publicly available. Can you reduce your project to a very simple example and provide it to us as an archive? I don't think we could reproduce your issue without it. You can even mail it to us if you don't want to share it here.
You can find our addresses in commits, e.g.
git show --compact-summary 2d709c6384102b73673b58d6509f8d5b1cb4840d
git show --compact-summary 5abb4f64551bd6410b2d96891dfe68e8304accf8@dobairoland @renzbagaporo
I think the problem has something to do with the kconfig tool or system. What I have noticed is that "Serial flasher config" values are getting screwed up every time, being reset to defaults?
make menuconfig tools to reflect the required values.make -j9 all the sdkconfig file will still be correct, untouched.idf.py build the sdkconfig will be changed and build will terminate with what seems like a valid error in one of my components, not being able to find "esp_tls.h" (still need help on solving this)idf.py menuconfig the sdkconfig will still be same as after step 2set(COMPONENTS "mbedtls") in the top level project CMakeLists.txt file (to try and solve the missing "esp_tls.h" problem) the sdkconfig gets completely changed.idf.py build with the changed sdkconfig file results in the Unknown CMake command "esptool_py_flash_target" error.idf.py build, still with the set(COMPONENTS "mbedtls") in the top level project CMakeLists.txt file, causes the build to very quickly terminate with the same Unknown CMake command "esptool_py_flash_target" error and again the sdkconfig is changed to the same as after step 4.Have attached the various sdkconfig files if this will help but seems the set(COMPONENTS "mbedtls") done based on #5215 (@igrr second last comment) is the cause of the problem.
sdkconfig 0 original.txt sdkconfig 1 after make all.txt sdkconfig 2 after idf py build.txt sdkconfig 3 After idf py menuconfig.txt sdkconfig 4 After set COMPONENTS mbedtls.txt sdkconfig 6 After idf py build with set COMPONENTS mbedtls.txt
ammaree
commented
4 years ago @dobairoland
Sorry, saw your comment after posting. Nope, do not have the project available publicly, and not going to be, it is a private commercial product. As far as finding the commits IDs is concerned, I am still lots on that. But I think it is maybe not relevant any more based on my previous post?
dobairoland
commented
4 years ago The commit IDs are still relevant so that we can narrow down the changes which might cause your issues.
Please don't switch between make & MSYS and idf.py & CMD because they are using a completely different kconfig management system. sdkconfigs should be compatible but (different and) better remove the sdkconfig file together with the build directory when switching. You can store your specific changes different than our default values in sdkconfig.default. see
ammaree
commented
4 years ago Only reason I switched between Make and CMake was to track when and what changes were made to sdkconfig since it resulted in an error where the FLASH size was changed from 4MB to 2MB (no idea why???) causing an error indicating that my custom partition could/would not fit, and terminating the build process. Currently on master showing
commit 1aebfdf6a69ff331d1160a564f46012e6bbe9187 (HEAD -> master, origin/master, origin/HEAD)
Merge: ad6056815 53316261b
Author: Krzysztof Budzynski <krzysztof@espressif.com>
Date: Fri May 1 04:43:56 2020 +0800
Merge branch 'bugfix/main_readme_s2' into 'master'
Mention S2 chips as well in the main README
See merge request espressif/esp-idf!8562
README.md | 4 ++--
README_CN.md | 6 ++----
2 files changed, 4 insertions(+), 6 deletions(-)I have no idea on how I would do the same for the commit I was using on 30 April...
dobairoland
commented
4 years ago @ammaree You could pick a much older commit and test it. If the picked commit still fails then pick another much older one (not a consequent one). When you hit a "good" commit then you can use git bisect to narrow down and find the exact commit which breaks your project.
renzbagaporo
commented
4 years ago @ammaree
Restoring to sdkconfig from step 0/1 and running idf.py build, still with the set(COMPONENTS "mbedtls") in the top level project CMakeLists.txt file, causes the build to very quickly terminate with the same Unknown CMake command "esptool_py_flash_target" error and again the sdkconfig is changed to the same as after step 4.
Using set(COMPONENTS trims down the build (as specified here: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/build-system.html#optional-project-variables), that is, you are specfiying to the build system to try and limit the components included in the project to the ones you specified and their dependencies (and down the chain). Note that @igrr's comment is in the context of trimming the build as well. esptool_py_flash_target lives in a component esptool_py, which might be causing it to be unspecified, since it's not being included in the build.
The effect you are seeing on the changing configs is because the excluded components config get removed as well.
Once I add set(COMPONENTS "mbedtls") in the top level project CMakeLists.txt file (to try and solve the missing "esp_tls.h" problem)
This seems to be the actual problem you want to solve (missing esp_tls.h). I propose reverting the use of set(COMPONENTS) to get rid of esptool_py_flash_target error, then we go from there.
Lastly, switching between make and cmake without a clean in between (purging 'existing' build directory, sdkconfig) is not something we encourage.
ammaree
commented
4 years ago @renzbagaporo Thanks for the feedback you are 100% correct. For now I have removed the set(COMPONENTS ????) line in order to first get the build process to finish successfully, will then start the optimization thereof.
My current obstacle revolve around errors whilst trying to include ESP IDF components, specifically #include "mbedtls/platform.h" and #include "spi_flash.h"
Example error is:
C:/Dropbox/devs/ws/z-components/x_sockets/x_sockets.h:34:11: fatal error: mbedtls/platform.h: No such file or directory
#include "mbedtls/platform.h"
^~~~~~~~~~~~~~~~~~~~Any suggestions
ammaree
commented
4 years ago @renzbagaporo @dobairoland
Have got the build down to failing only with multiple occurances of the exact same 3 inclusions being:
#include "mbedtls/platform.h" called from x_sockets.h with the CMakeLists.txt file
INCLUDE_DIRS "."
REQUIRES hal_esp32
PRIV_REQUIRES rtos-support common syslog printf systiming hal_esp32
)#include "esp_flash.h" called from hal_storage.h and #include "esp_spi_flash.h" called from hal_fota.h, both from same component with the CMakeLists.txt
hal_pwm.c hal_rtc.c hal_spi.c hal_storage.c hal_timer.c hal_udma.c hal_usart.c hal_watchdog.c
INCLUDE_DIRS .
REQUIRES common hal_esp32 http statistics
PRIV_REQUIRES buffers onewire pca9555 endpoints http identity irmacos rtos-support strings syslog systiming x_sockets http-parser libyuarel )The top level CMakeLists.txt file, although fairly long, is reasonably simple:
# ESP 32 IDF build system - IRMACS make file
cmake_minimum_required(VERSION 3.5)
set(MAKE_FILE_SEL 1)
set(BUILD_SEL D)
####################################################################################################
if (MAKE_FILE_SEL MATCHES 1) # AC0X Specification token
set(PROJECT_NAME 09d2b3e3-b034-4753-b1fe-13976a47bac4)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 2) # EM1P2 Specification token
set(PROJECT_NAME c77a5342-06ae-49eb-99e8-bf0d84da7829)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 6) # ESP-WROVER-KIT-V4 Specification token
set(PROJECT_NAME fc06ff52-9182-4256-8065-655330a57fe9)
set(ESP32_MODULE wrover4_8)
elseif (MAKE_FILE_SEL MATCHES 7) # M5 Stack Fire Specification token
set(PROJECT_NAME 5df0681b-df82-4682-851e-e289c7866dce)
set(ESP32_MODULE wrover16_4)
elseif (MAKE_FILE_SEL MATCHES 8) # DoIT DevKit V1 Specification token
set(PROJECT_NAME 80292bbd-3f32-48f7-affb-cc87db2ea0f2)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 10) # AC00 upstairs
set(PROJECT_NAME 240ac402d278)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 11) # WM-EM1P2 (2x1P) upstairs
set(PROJECT_NAME 807d3ac56254)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 12) # WM-EM3P2 (2x3P) upstairs
set(PROJECT_NAME 30aea432d390)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 13) # KSS-AC00 Shaun
set(PROJECT_NAME 240ac4036758)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 14) # WM-EM1P2 Shaun
set(PROJECT_NAME 30aea430ac14)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 15) # AC00 DemoKit
set(PROJECT_NAME 240ac4035c8c)
set(ESP32_MODULE wroom)
else()
set(PROJECT_NAME XXXXXXXXXXXX)
set(ESP32_MODULE wroom)
endif()
####################################################################################################
set(P_H esp32) # HAL directory selector
set(P_W C:/Dropbox/devs/ws) # WorkSpace top level directory
set(P_C ${P_W}/z-components)
set(P_S ${P_W}/z-support)
set(P_A ${P_W}/z-appl/irmacs) # Current application/project directory
set(P_B ${P_A}/build)
set(P_I ${P_W}/z-sdk/esp-idf)
if (BUILD_SEL MATCHES D)
set(SDK_BUILD ".debug")
elseif (BUILD_SEL MATCHES R)
set(SDK_BUILD ".release")
set(EXTRA_CFLAGS -Wno-unused-variable)
set(EXTRA_CPPFLAGS -Wno-unused-variable)
else()
set(SDK_BUILD "")
endif()
set(SDKCONFIG_DEFAULTS ${P_A}/sdkconfig.${ESP32_MODULE}${SDK_BUILD})
set(EXTRA_CFLAGS -D PROTOBUF_PACKED_STRUCT=1 -D PROTOBUF_BUFFER_ONLY=4 -Wno-implicit-fallthrough -Wno-format-truncation -Wno-deprecated-declarations -fstack-usage)
set(EXTRA_CPPFLAGS -D PROTOBUF_PACKED_STRUCT=1 -D PROTOBUF_BUFFER_ONLY=4 -Wno-implicit-fallthrough -Wno-format-truncation -Wno-deprecated-declarations -fstack-usage)
set(EXTRA_COMPONENT_DIRS "${P_C}/actuators ${P_C}/buffers ${P_C}/commands ${P_C}/common ${P_C}/devices/fonts ${P_C}/devices/ili9341 "
"${P_C}/devices/m90e26 ${P_C}/devices/onewire ${P_C}/devices/pca9555 ${P_C}/devices/ssd1306 ${P_C}/endpoints "
"${P_C}/hal_${P_H} ${P_C}/http ${P_C}/identity ${P_C}/irmacos ${P_C}/log ${P_C}/mqtt ${P_C}/printf ${P_C}/rtos-support "
"${P_C}/rules ${P_C}/sitewhere ${P_C}/sntp_client ${P_C}/statistics ${P_C}/strings ${P_C}/syslog ${P_C}/systiming "
"${P_C}/telnet_server ${P_C}/values ${P_C}/x_json ${P_C}/x_sockets ${P_S}/http-parser ${P_S}/libyuarel ${P_S}/nanopb" )
message("######################################################")
message("Project Name : " ${PROJECT_NAME})
message("ESP IDF location: " ${P_I})
message("Platform HAL : " ${P_H})
message("Path Application: " ${P_A})
message("Path Appl Build : " ${P_B})
message("Path Work Space : " ${P_W})
message("Path Components : " ${P_C})
message("Path Ext Support: " ${P_S})
message("SDKCONFIG file : " ${SDKCONFIG_DEFAULTS})
message("EXTRA_COMPONENT_DIRS: " ${EXTRA_COMPONENT_DIRS})
message("######################################################")
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(${PROJECT_NAME})Based on all the build documentation the mbedtls and spi_flash components must be automatically discovered, so what am I doing wrong.
I have now spent about 4 day on trying to convert a perfectly functional GNU Make project to CMake and just cannot believe that it can or should be this difficult. The full compile output is attached. Compile output.txt
ammaree
commented
4 years ago @renzbagaporo @dobairoland
On second thoughts, here is the complete output after having done a complete clean, the errors are the same but all the initial configuration info is also shown. Compile output.txt
ammaree
commented
4 years ago I think I have it fixed, will do last couple of files tomorrow, now need some sleep.
ammaree
commented
4 years ago @dobairoland @renzbagaporo Done, have it fixed and working. I will post the top-level CMakeLists.txt file here in case some of the things I had to do to get the full functionality working, might be of interest or value to someone else.
Thanks for your help, much appreciated.
# ESP 32 IDF build system - IRMACS make file
cmake_minimum_required(VERSION 3.5)
set(MAKE_FILE_SEL 1)
set(BUILD_SEL D)
####################################################################################################
if (MAKE_FILE_SEL MATCHES 1) # AC0X Specification token
set(PROJECT_NAME 09d2b3e3-b034-4753-b1fe-13976a47bac4)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 2) # EM1P2 Specification token
set(PROJECT_NAME c77a5342-06ae-49eb-99e8-bf0d84da7829)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 6) # ESP-WROVER-KIT-V4 Specification token
set(PROJECT_NAME fc06ff52-9182-4256-8065-655330a57fe9)
set(ESP32_MODULE wrover4_8)
elseif (MAKE_FILE_SEL MATCHES 7) # M5 Stack Fire Specification token
set(PROJECT_NAME 5df0681b-df82-4682-851e-e289c7866dce)
set(ESP32_MODULE wrover16_4)
elseif (MAKE_FILE_SEL MATCHES 8) # DoIT DevKit V1 Specification token
set(PROJECT_NAME 80292bbd-3f32-48f7-affb-cc87db2ea0f2)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 10) # AC00 upstairs
set(PROJECT_NAME 240ac402d278)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 11) # WM-EM1P2 (2x1P) upstairs
set(PROJECT_NAME 807d3ac56254)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 12) # WM-EM3P2 (2x3P) upstairs
set(PROJECT_NAME 30aea432d390)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 13) # KSS-AC00 Shaun
set(PROJECT_NAME 240ac4036758)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 14) # WM-EM1P2 Shaun
set(PROJECT_NAME 30aea430ac14)
set(ESP32_MODULE wroom)
elseif (MAKE_FILE_SEL MATCHES 15) # AC00 DemoKit
set(PROJECT_NAME 240ac4035c8c)
set(ESP32_MODULE wroom)
else()
set(PROJECT_NAME XXXXXXXXXXXX)
set(ESP32_MODULE wroom)
endif()
####################################################################################################
set(P_H esp32) # HAL directory selector
set(P_W C:/Dropbox/devs/ws) # WorkSpace top level directory
set(P_C ${P_W}/z-components)
set(P_S ${P_W}/z-support)
set(P_A ${P_W}/z-appl/irmacs) # Current application/project directory
set(P_B ${P_A}/build)
set(P_I ${P_W}/z-sdk/esp-idf)
if (BUILD_SEL MATCHES D)
set(SDK_BUILD ".debug")
elseif (BUILD_SEL MATCHES R)
set(SDK_BUILD ".release")
set(EXTRA_CFLAGS -Wno-unused-variable)
set(EXTRA_CPPFLAGS -Wno-unused-variable)
else()
set(SDK_BUILD "")
endif()
set(SDKCONFIG_DEFAULTS ${P_A}/sdkconfig.${ESP32_MODULE}${SDK_BUILD})
set(EXTRA_CFLAGS -D PROTOBUF_PACKED_STRUCT=1 -D PROTOBUF_BUFFER_ONLY=4 -Wno-implicit-fallthrough -Wno-format-truncation -Wno-deprecated-declarations -fstack-usage)
set(EXTRA_CPPFLAGS -D PROTOBUF_PACKED_STRUCT=1 -D PROTOBUF_BUFFER_ONLY=4 -Wno-implicit-fallthrough -Wno-format-truncation -Wno-deprecated-declarations -fstack-usage)
set(EXTRA_COMPONENT_DIRS "${P_C}/actuators ${P_C}/buffers ${P_C}/commands ${P_C}/common ${P_C}/devices/fonts ${P_C}/devices/ili9341 "
"${P_C}/devices/m90e26 ${P_C}/devices/onewire ${P_C}/devices/pca9555 ${P_C}/devices/ssd1306 ${P_C}/endpoints "
"${P_C}/hal_${P_H} ${P_C}/http ${P_C}/identity ${P_C}/irmacos ${P_C}/log ${P_C}/mqtt ${P_C}/printf ${P_C}/rtos-support "
"${P_C}/rules ${P_C}/sitewhere ${P_C}/sntp_client ${P_C}/statistics ${P_C}/strings ${P_C}/syslog ${P_C}/systiming "
"${P_C}/telnet_server ${P_C}/values ${P_C}/x_json ${P_C}/x_sockets"
"${P_S}/http-parser ${P_S}/libyuarel ${P_S}/nanopb ${P_S}/jsmn"
)
####################################################################################################
set( DATE_CMD C:/Dropbox/devs/bin/UnxUtils/usr/local/wbin/date.exe )
set( DATE_PAR +%s )
execute_process( COMMAND ${DATE_CMD} ${DATE_PAR} RESULT_VARIABLE DATE_RES OUTPUT_VARIABLE DATE_OUT OUTPUT_STRIP_TRAILING_WHITESPACE )
#message("Date Result: " ${DATE_RES} )
set( ECHO_CMD C:/Dropbox/devs/bin/UnxUtils/usr/local/wbin/echo.exe )
set( ECHO_PAR "#define APP_TIMESTAMP ${DATE_OUT}" )
set( ECHO_OUT ${P_C}/irmacos/timestamp.h )
execute_process( COMMAND ${ECHO_CMD} ${ECHO_PAR} RESULT_VARIABLE ECHO_RES OUTPUT_FILE ${ECHO_OUT} OUTPUT_STRIP_TRAILING_WHITESPACE )
#message("Echo Result: " ${ECHO_RES} )
set (GIT_CMD git )
set( GIT_PAR -C ${P_I}/components/esp_wifi/lib/esp32 rev-parse --short=8 HEAD )
execute_process( COMMAND ${GIT_CMD} ${GIT_PAR} RESULT_VARIABLE GIT_RES OUTPUT_VARIABLE GIT_OUT OUTPUT_STRIP_TRAILING_WHITESPACE )
set( EXTRA_CPPFLAGS -D WIFI_VER=${GIT_OUT} )
#message("Git Result : " ${GIT_RES} )
message("######################################################")
message("Project Name : " ${PROJECT_NAME} )
message("ESP IDF location: " ${P_I} )
message("Timestamp File : " ${ECHO_OUT} )
message("Build Timestamp : " ${DATE_OUT} )
message("WIFI FW Version : " ${GIT_OUT} )
message("Platform HAL : " ${P_H} )
message("Path Application: " ${P_A} )
message("Path Appl Build : " ${P_B} )
message("Path Work Space : " ${P_W} )
message("Path Components : " ${P_C} )
message("Path Ext Support: " ${P_S} )
message("SDKCONFIG file : " ${SDKCONFIG_DEFAULTS} )
message("EXTRA_COMPONENT_DIRS: " ${EXTRA_COMPONENT_DIRS} )
message("######################################################")
########################################################################## MUST BE RIGHT AT THE END
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(${PROJECT_NAME}) # Actual PROJECT name specificationI'm happy for your success @ammaree. I'm sorry that it took you so much effort to solve it.
Using CMake and ESP IDF master. As of 02h00 this morning CMake build was working with 3 day old master. I updated this morning to the latest master. A commit during the last 3 days is causing an error that was not present on the previous master from 3 days ago. The error message is as follows:
No changes were made other than the update. The exact same project is still building in latest MSYS without any errors.