search

platformio / platform-espressif32

Espressif 32: development platform for PlatformIO
https://registry.platformio.org/platforms/platformio/espressif32
Apache License 2.0
892 stars 600 forks source link

platform-espressif cannot satisfy python requirement for pyparsing >=2.0.3,<2.4.0 #462

Closed boxofrox closed 3 years ago

boxofrox commented 3 years ago

Synopsis

On ArchLinux and NixOS, the system package manager installs pyparsing-2.4.7. The espidf framework is unable to install an appropriate version of pyparsing. One user posted on the PlatformIO forum they tried to use venv to construct a python environment with an older version of pyparsing to no avail [1].

Are there any work arounds for this issue? My searches yielded few relevant results.

This error effectively makes platformio useless for my ESP32-related development. Would it make sense to incorporate patches that support pyparsing >2.4.0 in platform-espressif?

[1]: https://community.platformio.org/t/espressif32-python-dependencies/15839

Details

In a new, empty platformio project, I've set the following settings...

; platformio.ini

[env:usb]
board = esp32cam
platform = espressif32@^2.1.0
; platform = https://github.com/platformio/platform-espressif32.git
framework = espidf
lib_deps = espressif/esp32-camera@^1.0.0

...and when I run pio run, I find the following error:

# Snipped relevant section
The following Python requirements are not satisfied:
pyparsing>=2.0.3,<2.4.0
$ pio run
Processing usb (board: esp32cam; platform: espressif32@^2.1.0; framework: espidf)
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32cam.html
PLATFORM: Espressif 32 (2.1.0) > AI Thinker ESP32-CAM
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-espidf 3.40100.200827 (4.1.0)
 - tool-cmake 3.16.4
 - tool-esptoolpy 1.30000.201119 (3.0.0)
 - tool-ninja 1.7.1
 - toolchain-esp32ulp 1.22851.191205 (2.28.51)
 - toolchain-xtensa32 2.80200.200827 (8.2.0)
Reading CMake configuration...
-- Found Git: /usr/bin/git (found version "2.30.0")
-- The C compiler identification is GNU 8.2.0
-- The CXX compiler identification is GNU 8.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /home/devuser/.platformio/packages/toolchain-xtensa32/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /home/devuser/.platformio/packages/toolchain-xtensa32/bin/xtensa-esp32-elf-gcc
-- Check for working C compiler: /home/devuser/.platformio/packages/toolchain-xtensa32/bin/xtensa-esp32-elf-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/devuser/.platformio/packages/toolchain-xtensa32/bin/xtensa-esp32-elf-g++
-- Check for working CXX compiler: /home/devuser/.platformio/packages/toolchain-xtensa32/bin/xtensa-esp32-elf-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Project version: 4d8b3c9
-- Building ESP-IDF components for target esp32
-- Checking Python dependencies...
The following Python requirements are not satisfied:
pyparsing>=2.0.3,<2.4.0
Please follow the instructions found in the "Set up the tools" section of ESP-IDF Getting Started Guide
-- Configuring incomplete, errors occurred!
See also "/home/devuser/files/development/embedded/esp-cam/.pio/build/usb/CMakeFiles/CMakeOutput.log".

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
CMake Error at /home/devuser/.platformio/packages/framework-espidf/tools/cmake/build.cmake:271 (message):
  Some Python dependencies must be installed.  Check above message for
  details.
Call Stack (most recent call first):
  /home/devuser/.platformio/packages/framework-espidf/tools/cmake/build.cmake:397 (__build_check_python)
  /home/devuser/.platformio/packages/framework-espidf/tools/cmake/project.cmake:396 (idf_build_process)
  CMakeLists.txt:3 (project)
boxofrox commented 3 years ago

FYI. It appears the espressif team ran into a bug in pyparsing 2.4.0+ that prevented them from upgrading that dependency.

I've been at tearing at the code for at least four days trying to write an upgrade patch. I'm currently stuck at indentedBlock is broke and parsing beyond an indented region into the subsequent unindented region.

https://github.com/pyparsing/pyparsing/issues/87

valeros commented 3 years ago

Hi @boxofrox ! Could you please provide the contents of /home/devuser/files/development/embedded/esp-cam/.pio/build/usb/CMakeFiles/CMakeOutput.log ?

boxofrox commented 3 years ago

I placed the output of CMakeOutput.log in a gist [1]. The lines are fairly long and numerous, and seems easier to view the entire log with gist's raw file view.

The log file was generated by first uninstalling the platform-espressif32, removing my extra_scripts patches, and installing platform-espressif32 to return to unmodified, stock packages. Then ran pio run --verbose to kick off the build again, and verified the error matched the OP.

[1]: https://gist.github.com/boxofrox/0b799f7045cf6112dd2291eb62f5cfa4

boxofrox commented 3 years ago

Added output of pip freeze (ran in /home/devuser/files/development/embedded/esp-cam) to gist.

valeros commented 3 years ago

Is it a problem for you to reinstall the pyparsing package with any eligible version for ESP-IDF? As an alternative, you can install PlatformIO in an virtual environment which will be isolated from your system-wide installed packages.

boxofrox commented 3 years ago

Is it a problem for you to reinstall the pyparsing package with any eligible version for ESP-IDF?

I'm not sure. I figure since PlatformIO uses python, other tooling managed by PIO might also use python, and setting up a private python environment for ESP-IDF may cause other havoc for me at some point.

While I'm a polyglot programmer, I've avoided python, so I've got a bit of a learning curve to overcome. Will try to set up a python venv and a docker container and see what happens. I suspect the latter won't play well with uploading.

Thanks for taking a look.

valeros commented 3 years ago

You can use a universal installation script https://docs.platformio.org/en/latest/core/installation.html?highlight=virtual#step-2-install-platformio-core, so no need to mess with Docker.

boxofrox commented 3 years ago

@veleros Can confirm that the universal installation script was able to provide an isolated python + PlatformIO, and the espressif platform installed _pyparsing-2.3.1__ and compiled an "empty"/new project successfully.

FYI to anyone else searching for a solution.