search

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.51k stars 7.26k forks source link

idf.py menuconfig and build fails (IDFGH-2440) #4553

Closed mmurty closed 4 years ago

mmurty commented 4 years ago

Environment

Problem Description

I am unable to configure or build any of the example projects in the ESP-IDF. I am following the getting started guide on this link: https://docs.espressif.com/projects/esp-idf/en/latest/get-started/

I am installing all the tools using the ESP-IDF Tools installer from this link: https://dl.espressif.com/dl/esp-idf-tools-setup-2.2.exe

I select Python3.7 (also downloaded and installed by the installer itself). I have git installed version: 2.14.2.windows.1 The IDF git repository is also cloned by the installer itself. I selected release/v4.0. ( I have tried with almost all the branches mentioned here).

The installer installs every thing successfully and launches a cmd window image

After this I navigate to the D:\esp\examples\get-started\hello_world directory and run menuconfig by calling idf.py menuconfig. After I run this command cmake and ninja are executed and after some seconds the command line window crashes. There is no error displayed but several processes like python, ninja, cmake, xtensa compiler etc keep running in the background.

I have tried to capture the output of the command prompt by redirecting it to a file which I am posting below in the actual behavior section.

Expected Behavior

Should launch Menuconfig

Actual Behavior

Python requirements from D:\esp\requirements.txt are satisfied. Warn about uninitialized values. -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.14.2.windows.1") -- mconf-idf version mconf-v4.6.0.0-idf-20190628-win32 -- IDF_TARGET not set, using default target: esp32 -- The C compiler identification is GNU 8.2.0 -- The CXX compiler identification is GNU 8.2.0 -- The ASM compiler identification is GNU -- Found assembler: C:/Users/Mahesh/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe -- Check for working C compiler: C:/Users/Mahesh/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe -- Check for working C compiler: C:/Users/Mahesh/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc.exe -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: C:/Users/Mahesh/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++.exe -- Check for working CXX compiler: C:/Users/Mahesh/.espressif/tools/xtensa-esp32-elf/esp-2019r2-8.2.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++.exe -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Project version: v4.0-beta2-303-gb0d6c01b0 -- Building ESP-IDF components for target esp32 -- Found PythonInterp: C:/Users/Mahesh/.espressif/python_env/idf4.0_py3.7_env/Scripts/python.exe (found version "3.7.3") -- Could NOT find Perl (missing: PERL_EXECUTABLE) -- Adding linker script D:/esp/examples/get-started/hello_world/build/esp-idf/esp32/esp32_out.ld -- Adding linker script D:/esp/components/esp32/ld/esp32.project.ld.in -- Adding linker script D:/esp/components/esp32/ld/esp32.peripherals.ld -- Adding linker script D:/esp/components/esp_rom/esp32/ld/esp32.rom.ld -- Adding linker script D:/esp/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld -- Adding linker script D:/esp/components/esp_rom/esp32/ld/esp32.rom.syscalls.ld -- Adding linker script D:/esp/components/esp_rom/esp32/ld/esp32.rom.newlib-data.ld -- Adding linker script D:/esp/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld -- Components: app_trace app_update asio bootloader bootloader_support bt coap console cxx driver efuse esp-tls esp32 esp_adc_cal esp_common esp_eth esp_event esp_gdbstub esp_http_client esp_http_server esp_https_ota esp_https_server esp_local_ctrl esp_ringbuf esp_rom esp_websocket_client esp_wifi espcoredump esptool_py expat fatfs freemodbus freertos heap idf_test jsmn json libsodium log lwip main mbedtls mdns mqtt newlib nghttp nvs_flash openssl partition_table protobuf-c protocomm pthread sdmmc soc spi_flash spiffs tcp_transport tcpip_adapter ulp unity vfs wear_levelling wifi_provisioning wpa_supplicant xtensa -- Component paths: D:/esp/components/app_trace D:/esp/components/app_update D:/esp/components/asio D:/esp/components/bootloader D:/esp/components/bootloader_support D:/esp/components/bt D:/esp/components/coap D:/esp/components/console D:/esp/components/cxx D:/esp/components/driver D:/esp/components/efuse D:/esp/components/esp-tls D:/esp/components/esp32 D:/esp/components/esp_adc_cal D:/esp/components/esp_common D:/esp/components/esp_eth D:/esp/components/esp_event D:/esp/components/esp_gdbstub D:/esp/components/esp_http_client D:/esp/components/esp_http_server D:/esp/components/esp_https_ota D:/esp/components/esp_https_server D:/esp/components/esp_local_ctrl D:/esp/components/esp_ringbuf D:/esp/components/esp_rom D:/esp/components/esp_websocket_client D:/esp/components/esp_wifi D:/esp/components/espcoredump D:/esp/components/esptool_py D:/esp/components/expat D:/esp/components/fatfs D:/esp/components/freemodbus D:/esp/components/freertos D:/esp/components/heap D:/esp/components/idf_test D:/esp/components/jsmn D:/esp/components/json D:/esp/components/libsodium D:/esp/components/log D:/esp/components/lwip D:/esp/examples/get-started/hello_world/main D:/esp/components/mbedtls D:/esp/components/mdns D:/esp/components/mqtt D:/esp/components/newlib D:/esp/components/nghttp D:/esp/components/nvs_flash D:/esp/components/openssl D:/esp/components/partition_table D:/esp/components/protobuf-c D:/esp/components/protocomm D:/esp/components/pthread D:/esp/components/sdmmc D:/esp/components/soc D:/esp/components/spi_flash D:/esp/components/spiffs D:/esp/components/tcp_transport D:/esp/components/tcpip_adapter D:/esp/components/ulp D:/esp/components/unity D:/esp/components/vfs D:/esp/components/wear_levelling D:/esp/components/wifi_provisioning D:/esp/components/wpa_supplicant D:/esp/components/xtensa -- Configuring done -- Generating done -- Build files have been written to: D:/esp/examples/get-started/hello_world/build [0/1] cmd.exe /C "cd /D D:\esp\examples\get-started\hello_world\build && C:\Users\Mahesh.espressif\python_env\idf4.0_py3.7_env\Scripts\python.exe D:/esp/tools/kconfig_new/confgen.py --kconfig D:/esp/Kconfig --sdkconfig-rename D:/esp/sdkconfig.rename --config D:/esp/examples/get-started/hello_world/sdkconfig --env-file D:/esp/examples/get-started/hello_world/build/config.env --env IDF_TARGET=esp32 --output config D:/esp/examples/get-started/hello_world/sdkconfig && C:\Users\Mahesh.espressif\tools\cmake\3.13.4\bin\cmake.exe -E env "COMPONENT_KCONFIGS=D:/esp/components/app_trace/Kconfig D:/esp/components/bt/Kconfig D:/esp/components/driver/Kconfig D:/esp/components/efuse/Kconfig D:/esp/components/esp-tls/Kconfig D:/esp/components/esp32/Kconfig D:/esp/components/esp_adc_cal/Kconfig D:/esp/components/esp_common/Kconfig D:/esp/components/esp_eth/Kconfig D:/esp/components/esp_event/Kconfig D:/esp/components/esp_gdbstub/Kconfig D:/esp/components/esp_http_client/Kconfig D:/esp/components/esp_http_server/Kconfig D:/esp/components/esp_https_ota/Kconfig D:/esp/components/esp_https_server/Kconfig D:/esp/components/esp_wifi/Kconfig D:/esp/components/espcoredump/Kconfig D:/esp/components/fatfs/Kconfig D:/esp/components/freemodbus/Kconfig D:/esp/components/freertos/Kconfig D:/esp/components/heap/Kconfig D:/esp/components/libsodium/Kconfig D:/esp/components/log/Kconfig D:/esp/components/lwip/Kconfig D:/esp/components/mbedtls/Kconfig D:/esp/components/mdns/Kconfig D:/esp/components/mqtt/Kconfig D:/esp/components/newlib/Kconfig D:/esp/components/nvs_flash/Kconfig D:/esp/components/openssl/Kconfig D:/esp/components/pthread/Kconfig D:/esp/components/spi_flash/Kconfig D:/esp/components/spiffs/Kconfig D:/esp/components/tcpip_adapter/Kconfig D:/esp/components/unity/Kconfig D:/esp/components/vfs/Kconfig D:/esp/components/wear_levelling/Kconfig D:/esp/components/wifi_provisioning/Kconfig D:/esp/components/wpa_supplicant/Kconfig" "COMPONENT_KCONFIGS_PROJBUILD=D:/esp/components/app_update/Kconfig.projbuild D:/esp/components/bootloader/Kconfig.projbuild D:/esp/components/esptool_py/Kconfig.projbuild D:/esp/components/partition_table/Kconfig.projbuild" IDF_CMAKE=y KCONFIG_CONFIG=D:/esp/examples/get-started/hello_world/sdkconfig IDF_TARGET=esp32 C:/Users/Mahesh/.espressif/tools/mconf/v4.6.0.0-idf-20190628/mconf-idf.exe D:/esp/Kconfig && C:\Users\Mahesh.espressif\python_env\idf4.0_py3.7_env\Scripts\python.exe D:/esp/tools/kconfig_new/confgen.py --kconfig D:/esp/Kconfig --sdkconfig-rename D:/esp/sdkconfig.rename --config D:/esp/examples/get-started/hello_world/sdkconfig --env-file D:/esp/examples/get-started/hello_world/build/config.env --env IDF_TARGET=esp32 --output config D:/esp/examples/get-started/hello_world/sdkconfig" FAILED: CMakeFiles/menuconfig cmd.exe /C "cd /D D:\esp\examples\get-started\hello_world\build && C:\Users\Mahesh.espressif\python_env\idf4.0_py3.7_env\Scripts\python.exe D:/esp/tools/kconfig_new/confgen.py --kconfig D:/esp/Kconfig --sdkconfig-rename D:/esp/sdkconfig.rename --config D:/esp/examples/get-started/hello_world/sdkconfig --env-file D:/esp/examples/get-started/hello_world/build/config.env --env IDF_TARGET=esp32 --output config D:/esp/examples/get-started/hello_world/sdkconfig && C:\Users\Mahesh.espressif\tools\cmake\3.13.4\bin\cmake.exe -E env "COMPONENT_KCONFIGS=D:/esp/components/app_trace/Kconfig D:/esp/components/bt/Kconfig D:/esp/components/driver/Kconfig D:/esp/components/efuse/Kconfig D:/esp/components/esp-tls/Kconfig D:/esp/components/esp32/Kconfig D:/esp/components/esp_adc_cal/Kconfig D:/esp/components/esp_common/Kconfig D:/esp/components/esp_eth/Kconfig D:/esp/components/esp_event/Kconfig D:/esp/components/esp_gdbstub/Kconfig D:/esp/components/esp_http_client/Kconfig D:/esp/components/esp_http_server/Kconfig D:/esp/components/esp_https_ota/Kconfig D:/esp/components/esp_https_server/Kconfig D:/esp/components/esp_wifi/Kconfig D:/esp/components/espcoredump/Kconfig D:/esp/components/fatfs/Kconfig D:/esp/components/freemodbus/Kconfig D:/esp/components/freertos/Kconfig D:/esp/components/heap/Kconfig D:/esp/components/libsodium/Kconfig D:/esp/components/log/Kconfig D:/esp/components/lwip/Kconfig D:/esp/components/mbedtls/Kconfig D:/esp/components/mdns/Kconfig D:/esp/components/mqtt/Kconfig D:/esp/components/newlib/Kconfig D:/esp/components/nvs_flash/Kconfig D:/esp/components/openssl/Kconfig D:/esp/components/pthread/Kconfig D:/esp/components/spi_flash/Kconfig D:/esp/components/spiffs/Kconfig D:/esp/components/tcpip_adapter/Kconfig D:/esp/components/unity/Kconfig D:/esp/components/vfs/Kconfig D:/esp/components/wear_levelling/Kconfig D:/esp/components/wifi_provisioning/Kconfig D:/esp/components/wpa_supplicant/Kconfig" "COMPONENT_KCONFIGS_PROJBUILD=D:/esp/components/app_update/Kconfig.projbuild D:/esp/components/bootloader/Kconfig.projbuild D:/esp/components/esptool_py/Kconfig.projbuild D:/esp/components/partition_table/Kconfig.projbuild" IDF_CMAKE=y KCONFIG_CONFIG=D:/esp/examples/get-started/hello_world/sdkconfig IDF_TARGET=esp32 C:/Users/Mahesh/.espressif/tools/mconf/v4.6.0.0-idf-20190628/mconf-idf.exe D:/esp/Kconfig && C:\Users\Mahesh.espressif\python_env\idf4.0_py3.7_env\Scripts\python.exe D:/esp/tools/kconfig_new/confgen.py --kconfig D:/esp/Kconfig --sdkconfig-rename D:/esp/sdkconfig.rename --config D:/esp/examples/get-started/hello_world/sdkconfig --env-file D:/esp/examples/get-started/hello_world/build/config.env --env IDF_TARGET=esp32 --output config D:/esp/examples/get-started/hello_world/sdkconfig" ninja: build stopped: subcommand failed. Checking Python dependencies... Executing action: menuconfig Running cmake in directory d:\esp\examples\get-started\hello_world\build Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 --warn-uninitialized -DCCACHE_ENABLE=0 d:\esp\examples\get-started\hello_world"... Running ninja in directory d:\esp\examples\get-started\hello_world\build Executing "ninja menuconfig"... ninja failed with exit code 1

Steps to repropduce

  1. Download tools installer from https://dl.espressif.com/dl/esp-idf-tools-setup-2.2.exe
  2. Double click and launch the installer. image image image image

After this installation completes successfully and launches the cmd prompt.

mmurty commented 4 years ago

I forgot to mention: I have earlier developed for ESP32 using the MSYS environment. That worked for me and is already installed. I did not delete it and I am trying to install the latest tools.

dobairoland commented 4 years ago

Hi @mmurty. Thank you for reporting the issue. I've never seen anything like this before.

Could you please go into D:\esp\examples\get-started\hello_world and run the following command:

cmake -E env "COMPONENT_CONFIGS= " "COMPONENT_KCONFIGS_PROJBUILD= " "COMPONENT_KCONFIGS= " "IDF_CMAKE=y" "IDF_TARGET=esp32" mconf-idf D:/esp/Kconfig

Does this command breaks your Windows command line?

dobairoland commented 4 years ago

Does idf.py build work for you?

mmurty commented 4 years ago

@dobairoland No idf.py build doesn't work. Same thing happens. The terminal crashes after some time. And yes I am able to see the menuconfig screen when I run the cmake -E env "COMPONENT_CONFIGS= " "COMPONENT_KCONFIGS_PROJBUILD= " "COMPONENT_KCONFIGS= " "IDF_CMAKE=y" "IDF_TARGET=esp32" mconf-idf D:/esp/Kconfig command.

mmurty commented 4 years ago

I have tried changing installation paths, IDF versions, latest python vs old python etc and other combinations but it doesn't work. I earlier had chocolatey installed which installed cmake and ninja. Suspecting that it is creating issues I uninstalled it. Also I uninstalled cmake, ninja, python and other tools which I suspect may be creating problems / clashes with the ESP-IDF. Now I download the new python 3.7 environment through the tools installer and install everything in the %userprofile% path. (Both .espressif folder and the esp-idf). Still the issue persists. Although I also downloaded the latest MSYS32 environment with IDF 3.3 version and I am able to build with it. As it is too slow I would like to have the CMAKE environment running.

dobairoland commented 4 years ago

Yes, probably something is broken in your system. I'm sorry for asking you to run further commands but I can only guess what could cause the problem.

Please try this:

 C:\Users\Mahesh.espressif\python_env\idf4.0_py3.7_env\Scripts\python D:\esp\tools\kconfig_new\confgen.py --kconfig D:\esp\Kconfig --config sdkconfig --env-file build\config.env --env IDF_TARGET=esp32 --output config sdkconfig

At what point is the following command fail? Immediately or later?

cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 --warn-uninitialized -DCCACHE_ENABLE=0 d:\esp\examples\get-started\hello_world

Please also review the content of your PATH environment variable. Maybe an item there pointing to a broken program.

mmurty commented 4 years ago

@dobairoland When I run this command C:\Users\Mahesh.espressif\python_env\idf4.0_py3.7_env\Scripts\python D:\esp\tools\kconfig_new\confgen.py --kconfig D:\esp\Kconfig --config sdkconfig --env-file build\config.env --env IDF_TARGET=esp32 --output config sdkconfig I do not get any output and the console window does not exit. But when I run this command it displays some output and breaks immediately. cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 --warn-uninitialized -DCCACHE_ENABLE=0 d:\esp\examples\get-started\hello_world I will try to capture the output and attach it here.

dobairoland commented 4 years ago

Hello @mmurty. Do you have any update regarding your issue?

mmurty commented 4 years ago

@dobairoland I tried to capture the output but I could not do it. Where do you guess may be the issue? As mentioned earlier as soon as I run the command the cmd window dissappears and there are several tasks running in my task manager.

dobairoland commented 4 years ago

@mmurty I'm sorry but I have no idea. I asked you to run those commands to narrow down the problem. A lot of different processes are started during the build. Any of those executables could be broken. Or it could be an issue with the Windows scheduler so the failure occurs only when more of them are running and nothing fails when started individually.

pacucha42 commented 4 years ago

Hi @mmurty, the issue of "crashing" is quite likely caused by sh.exe (or some of its params) installed with your Git version. Our build system runs few git commands in the beginning which launch a long chain of git-internal processes incl sh.exe (eg for getting submodule status). One of these processes closes the cmd console window BUT not the process chain itself => the build continues in the background, that's why you see the output continues in your redirected log (no crash). This applies to both idf.py menuconfig and idf.py build However, I couldn't figure out exact cause of such behavior, sorry :-(

The solution is to upgrade your Git to the most recent version (2.25.0 at the time of writing), tested repeatedly - please, try and let me know. (you can install the newest Git over your existing version, ESP-IDF scripts shall figure it out automatically)

mmurty commented 4 years ago

@pacucha42 Thanks for the suggestion. I updated GIT to version 2.25.0 and everything is working fine. now. Earlier I thought of updating GIT but I neglected it thinking it may not be causing the issue.