[Docker] c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory

[soxhi8]

The problem

My freshly installed ESPHome-host cannot compile firmware due to a missing dependency. Host is a Raspberry Pi 4 (Linux hostname 5.10.103-v8+ #1529 SMP PREEMPT Tue Mar 8 12:26:46 GMT 2022 aarch64 GNU/Linux).

Which version of ESPHome has the issue?

2023.8.3

What type of installation are you using?

Docker

Which version of Home Assistant has the issue?

No response

What platform are you using?

ESP32-IDF

Board

ESP32-WROOM-32D

Component causing the issue

ffi.h

Example YAML snippet

substitutions:
  name: esp32-bluetooth-proxy-62d4d4
  friendly_name: Bluetooth Proxy 62d4d4
packages:
  esphome.bluetooth-proxy: github://esphome/bluetooth-proxies/esp32-generic.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
esp32:
  board: esp32dev
  framework:
    type: esp-idf
api:
  encryption:
    key: [redacted]

ota:

logger:

esp32_ble_tracker:
  scan_parameters:
    interval: 1100ms
    window: 1100ms
    active: true

bluetooth_proxy:
  active: true

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

Anything in the logs that might be useful for us?

INFO ESPHome 2023.8.3
INFO Reading configuration /config/esp32-bluetooth-proxy-62d4d4.yaml...
INFO Generating C++ source...
INFO Compiling app...
Processing esp32-bluetooth-proxy-62d4d4 (board: esp32dev; framework: espidf; platform: platformio/espressif32@5.4.0)
--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - framework-espidf @ 3.40405.230623 (4.4.5) 
 - tool-cmake @ 3.16.9 
 - tool-ninja @ 1.10.2 
 - toolchain-esp32ulp @ 2.35.0-20220830 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Installing ESP-IDF's Python dependencies
Collecting urllib3<2
  Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Collecting cryptography<35.0.0,>=2.1.4
  Using cached cryptography-3.4.8.tar.gz (546 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /root/.platformio/penv/.espidf-4.4.5/bin/python /root/.platformio/penv/.espidf-4.4.5/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-lmziftr2/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4'
       cwd: None
  Complete output (62 lines):
  Collecting setuptools>=40.6.0
    Using cached setuptools-68.2.0-py3-none-any.whl (807 kB)
  Collecting wheel
    Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
  Collecting cffi>=1.12
    Using cached cffi-1.15.1.tar.gz (508 kB)
  Collecting setuptools-rust>=0.11.4
    Using cached setuptools_rust-1.7.0-py3-none-any.whl (25 kB)
  Collecting pycparser
    Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
  Collecting typing-extensions>=3.7.4.3
    Using cached typing_extensions-4.7.1-py3-none-any.whl (33 kB)
  Collecting semantic-version<3,>=2.8.2
    Using cached semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
  Collecting tomli>=1.2.1
    Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
  Using legacy 'setup.py install' for cffi, since package 'wheel' is not installed.
  Installing collected packages: typing-extensions, tomli, setuptools, semantic-version, pycparser, wheel, setuptools-rust, cffi
      Running setup.py install for cffi: started
      Running setup.py install for cffi: finished with status 'error'
      ERROR: Command errored out with exit status 1:
       command: /root/.platformio/penv/.espidf-4.4.5/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h1tfygxl/cffi_e8d02f7a80664963a252ba60607d17d5/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h1tfygxl/cffi_e8d02f7a80664963a252ba60607d17d5/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2qs69g4g/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-lmziftr2/overlay --compile --install-headers /tmp/pip-build-env-lmziftr2/overlay/include/site/python3.9/cffi
           cwd: /tmp/pip-install-h1tfygxl/cffi_e8d02f7a80664963a252ba60607d17d5/
      Complete output (36 lines):
      running install
      running build
      running build_py
      creating build
      creating build/lib.linux-aarch64-3.9
      creating build/lib.linux-aarch64-3.9/cffi
      copying cffi/model.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/recompiler.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/cparser.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/cffi_opcode.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/api.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/backend_ctypes.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/__init__.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/ffiplatform.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/pkgconfig.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/lock.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/vengine_gen.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/error.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/setuptools_ext.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/commontypes.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/verifier.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/vengine_cpy.py -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/_cffi_include.h -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/parse_c_type.h -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/_embedding.h -> build/lib.linux-aarch64-3.9/cffi
      copying cffi/_cffi_errors.h -> build/lib.linux-aarch64-3.9/cffi
      running build_ext
      building '_cffi_backend' extension
      creating build/temp.linux-aarch64-3.9
      creating build/temp.linux-aarch64-3.9/c
      arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/.platformio/penv/.espidf-4.4.5/include -I/usr/include/python3.9 -c c/_cffi_backend.c -o build/temp.linux-aarch64-3.9/c/_cffi_backend.o
      c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory

*************************************************************
* Looking for ffi.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:ffi.h"
* Web  > https://registry.platformio.org/search?q=header:ffi.h
*
*************************************************************

         15 | #include <ffi.h>
            |          ^~~~~~~
      compilation terminated.
      error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1
      ----------------------------------------
  ERROR: Command errored out with exit status 1: /root/.platformio/penv/.espidf-4.4.5/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h1tfygxl/cffi_e8d02f7a80664963a252ba60607d17d5/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h1tfygxl/cffi_e8d02f7a80664963a252ba60607d17d5/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2qs69g4g/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-lmziftr2/overlay --compile --install-headers /tmp/pip-build-env-lmziftr2/overlay/include/site/python3.9/cffi Check the logs for full command output.
  ----------------------------------------

Additional information

No response

[timball]

i am getting the same compilation error on the same hardware and similar esp32 board.

last version of esphome this board was able compile under was 2023.8.3

--timball

[tigers75]

Can confirm the problem on Raspi4 via docker. I can compile without problems for a generic esp32 with BT proxy on, only compiling for shelly (at least for the shelly1plus - only one I tried) fails.

edit: I have no BLE_tracker option Edit2: even disabling Bluetooth proxy and trying to build the basic shelly 1 plus template fails, must be a problem with non Arduino platform

[rvanderven]

I experienced the same with esphome 2023.8.3

[yisonPylkita]

Same issue with M5Stack AtomS3 Lite (ESP32-S3)

[tigers75]

I tried manually installing ffi inside the docker container using:

docker exec apt-get update -y docker exec apt-get install -y libffi-dev

And using the latest beta of ESPHome now I get a different error, maybe someone can get some more info from it?

`Building wheels for collected packages: cryptography, msgpack, pyyaml Building wheel for cryptography (PEP 517): started Building wheel for cryptography (PEP 517): finished with status 'error' ERROR: Command errored out with exit status 1: command: /root/.platformio/penv/.espidf-4.4.5/bin/python /tmp/tmpibr1zwkx_in_process.py buildwheel /tmp/tmp8l8kdx0e cwd: /tmp/pip-install-sua2y2m/cryptography_0495ed3a401143ddad1e70213f1c0c77 Complete output (165 lines): running bdist_wheel running build running build_py creating build creating build/lib.linux-armv7l-cpython-39 creating build/lib.linux-armv7l-cpython-39/cryptography copying src/cryptography/fernet.py -> build/lib.linux-armv7l-cpython-39/cryptography copying src/cryptography/about.py -> build/lib.linux-armv7l-cpython-39/cryptography copying src/cryptography/exceptions.py -> build/lib.linux-armv7l-cpython-39/cryptography copying src/cryptography/utils.py -> build/lib.linux-armv7l-cpython-39/cryptography copying src/cryptography/init.py -> build/lib.linux-armv7l-cpython-39/cryptography creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat copying src/cryptography/hazmat/_types.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat copying src/cryptography/hazmat/_oid.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat copying src/cryptography/hazmat/_der.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat copying src/cryptography/hazmat/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat creating build/lib.linux-armv7l-cpython-39/cryptography/x509 copying src/cryptography/x509/general_name.py -> build/lib.linux-armv7l-cpython-39/cryptography/x509 copying src/cryptography/x509/certificate_transparency.py -> build/lib.linux-armv7l-cpython-39/cryptography/x509 copying src/cryptography/x509/name.py -> build/lib.linux-armv7l-cpython-39/cryptography/x509 Building wheel for msgpack (PEP 517): started copying src/cryptography/x509/extensions.py -> build/lib.linux-armv7l-cpython-39/cryptography/x509 copying src/cryptography/x509/ocsp.py -> build/lib.linux-armv7l-cpython-39/cryptography/x509 copying src/cryptography/x509/base.py -> build/lib.linux-armv7l-cpython-39/cryptography/x509 copying src/cryptography/x509/oid.py -> build/lib.linux-armv7l-cpython-39/cryptography/x509 copying src/cryptography/x509/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/x509 creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/bindings copying src/cryptography/hazmat/bindings/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/bindings creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends copying src/cryptography/hazmat/backends/interfaces.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends copying src/cryptography/hazmat/backends/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/hmac.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/constant_time.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/poly1305.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/_asymmetric.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/_cipheralgorithm.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/hashes.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/keywrap.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/padding.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/_serialization.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/cmac.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/bindings/openssl copying src/cryptography/hazmat/bindings/openssl/_conditional.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/bindings/openssl copying src/cryptography/hazmat/bindings/openssl/binding.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/bindings/openssl copying src/cryptography/hazmat/bindings/openssl/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/bindings/openssl creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/hmac.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ciphers.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/aead.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/x509.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/backend.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/poly1305.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/dh.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/encode_asn1.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/decode_asn1.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/x448.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/rsa.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ed448.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ec.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/dsa.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/hashes.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/utils.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/x25519.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ed25519.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ocsp.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/cmac.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/backends/openssl creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/aead.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/algorithms.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/modes.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/base.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/ciphers creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/concatkdf.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/scrypt.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/x963kdf.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/pbkdf2.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/kbkdf.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/hkdf.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/kdf creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/twofactor copying src/cryptography/hazmat/primitives/twofactor/utils.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/twofactor copying src/cryptography/hazmat/primitives/twofactor/totp.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/twofactor copying src/cryptography/hazmat/primitives/twofactor/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/twofactor copying src/cryptography/hazmat/primitives/twofactor/hotp.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/twofactor creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/dh.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/x448.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/rsa.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/ed448.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/ec.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/dsa.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/utils.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/x25519.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/padding.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/ed25519.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/asymmetric creating build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/pkcs12.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/ssh.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/base.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/pkcs7.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/init.py -> build/lib.linux-armv7l-cpython-39/cryptography/hazmat/primitives/serialization running egg_info writing src/cryptography.egg-info/PKG-INFO writing dependency_links to src/cryptography.egg-info/dependency_links.txt writing requirements to src/cryptography.egg-info/requires.txt writing top-level names to src/cryptography.egg-info/top_level.txt reading manifest file 'src/cryptography.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' no previously-included directories found matching 'docs/_build' warning: no previously-included files found matching 'vectors' warning: no previously-included files matching '' found under directory 'vectors' warning: no previously-included files matching '' found under directory '.github' warning: no previously-included files found matching 'release.py' warning: no previously-included files found matching '.coveragerc' warning: no previously-included files found matching 'codecov.yml' warning: no previously-included files found matching '.readthedocs.yml' warning: no previously-included files found matching 'dev-requirements.txt' warning: no previously-included files found matching 'tox.ini' warning: no previously-included files found matching 'mypy.ini' warning: no previously-included files matching '' found under directory '.zuul.d' warning: no previously-included files matching '' found under directory '.zuul.playbooks' adding license file 'LICENSE' adding license file 'LICENSE.APACHE' adding license file 'LICENSE.BSD' adding license file 'LICENSE.PSF' writing manifest file 'src/cryptography.egg-info/SOURCES.txt' copying src/cryptography/py.typed -> build/lib.linux-armv7l-cpython-39/cryptography running build_ext generating cffi module 'build/temp.linux-armv7l-cpython-39/_padding.c' creating build/temp.linux-armv7l-cpython-39 generating cffi module 'build/temp.linux-armv7l-cpython-39/_openssl.c' running build_rust

  =============================DEBUG ASSISTANCE=============================
  If you are seeing a compilation error please try the following steps to
  successfully install cryptography:
  1) Upgrade to the latest pip and try again. This will fix errors for most
     users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
  2) Read https://cryptography.io/en/latest/installation.html for specific
     instructions for your platform.
  3) Check our frequently asked questions for more information:
     https://cryptography.io/en/latest/faq.html
  4) Ensure you have a recent Rust toolchain installed:
     https://cryptography.io/en/latest/installation.html#rust
  5) If you are experiencing issues with Rust for *this release only* you may
     set the environment variable `CRYPTOGRAPHY_DONT_BUILD_RUST=1`.
  =============================DEBUG ASSISTANCE=============================

error: can't find Rust compiler

If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.

To update pip, run:

  pip install --upgrade pip

and then retry package installation.

If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.r s) is the recommended way to download and update the Rust compiler toolchain.

This package requires Rust >=1.41.0.

ERROR: Failed building wheel for cryptography Building wheel for msgpack (PEP 517): finished with status 'done' Created wheel for msgpack: filename=msgpack-1.0.6-cp39-cp39-linux_armv7l.whl size=487468 sha256=b7a84cd5b2e2949d9764ee92d701b0d33891059998be20d1742fa8fc7d30b0b4 Stored in directory: /root/.cache/pip/wheels/37/4e/d3/fa9d02a88faaceddfc21a594ad8673ba36dd2766c21bee55b6 Building wheel for pyyaml (PEP 517): started Building wheel for pyyaml (PEP 517): finished with status 'done' Created wheel for pyyaml: filename=PyYAML-6.0.1-cp39-cp39-linux_armv7l.whl size=45359 sha256=418e9d5e260d7c22ec3c82dbb7e2e02b33efb1e64967ae74e581505b8eda1cb6 Stored in directory: /root/.cache/pip/wheels/1c/ff/88/3a317d1db793206ab64cb66367bf6f84186b75849656be786a Successfully built msgpack pyyaml Failed to build cryptography ERROR: Could not build wheels for cryptography which use PEP 517 and cannot be installed directly *** Error 1 Reading CMake configuration... -- Found Git: /usr/bin/git (found version "2.30.2") -- The C compiler identification is GNU 8.4.0 -- The CXX compiler identification is GNU 8.4.0 -- The ASM compiler identification is GNU -- Found assembler: /root/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- Check for working C compiler: /root/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- Check for working C compiler: /root/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- broken -- Configuring incomplete, errors occurred! See also "/config/.esphome/build/esp32-bluetooth-proxy/.pioenvs/esp32-bluetooth-proxy/CMakeFiles/CMakeOutput.log". See also "/config/.esphome/build/esp32-bluetooth-proxy/.pioenvs/esp32-bluetooth-proxy/CMakeFiles/CMakeError.log".

fatal: not a git repository (or any of the parent directories): .git CMake Error at /root/.platformio/packages/tool-cmake/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake:60 (message): The C compiler

"/root/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: /config/.esphome/build/esp32-bluetooth-proxy/.pioenvs/esp32-bluetooth-proxy/CMakeFiles/CMakeTmp

Run Build Command(s):/root/.platformio/packages/tool-ninja/ninja cmTC_9f803 && [1/2] Building C object CMakeFiles/cmTC_9f803.dir/testCCompiler.c.obj
[2/2] Linking C executable cmTC_9f803
FAILED: cmTC_9f803
: && /root/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -mlongcalls -Wno-frame-address   CMakeFiles/cmTC_9f803.dir/testCCompiler.c.obj  -o cmTC_9f803   && :
/root/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /root/.platformio/packages/toolchain-xtensa-esp32/bin/../libexec/gcc/xtensa-esp32-elf/8.4.0/      liblto_plugin.so: error loading plugin: /root/.platformio/packages/toolchain-xtensa-esp32/bin/../libexec/gcc/xtensa-esp32-elf/8.4.0/liblto_plugin.so: cannot open shared object file: No such file or directory
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

CMake will not be able to correctly generate this project. Call Stack (most recent call first): /root/.platformio/packages/framework-espidf/tools/cmake/project.cmake:298 (__project) CMakeLists.txt:3 (project) `

[ssieb]

Will probably be fixed by https://github.com/esphome/esphome/pull/5450 or https://github.com/esphome/esphome/pull/5473

[darraghbr]

Unfortunately I am still getting this same error, running esphome 2023.10.1 addon through Home Assistant, I used the ESPHome Ready-Made Projects installer through chrome to make a Bluetooth Proxy. It told me that it was installing 2023.10.1 onto the Olimex ESP32 PoE ISO. It installs fine and then when I adopt it through the ESPHome addon I get this.

INFO ESPHome 2023.10.1
INFO Reading configuration /config/esphome/olimex-bluetooth-proxy-3af624.yaml...
WARNING GPIO12 is a Strapping PIN and should be avoided.
Attaching external pullup/down resistors to strapping pins can cause unexpected failures.
See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins
INFO Generating C++ source...
INFO Compiling app...
Processing olimex-bluetooth-proxy-3af624 (board: esp32-poe-iso; framework: espidf; platform: platformio/espressif32@5.4.0)
--------------------------------------------------------------------------------
Library Manager: Installing esphome/noise-c @ 0.1.4
INFO Installing esphome/noise-c @ 0.1.4
Unpacking  [####################################]  100%
Library Manager: noise-c@0.1.4 has been installed!
INFO noise-c@0.1.4 has been installed!
Library Manager: Resolving dependencies...
INFO Resolving dependencies...
Library Manager: Installing esphome/libsodium @ 1.10018.1
INFO Installing esphome/libsodium @ 1.10018.1
Unpacking  [####################################]  100%
Library Manager: libsodium@1.10018.1 has been installed!
INFO libsodium@1.10018.1 has been installed!
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - framework-espidf @ 3.40405.230623 (4.4.5) 
 - tool-cmake @ 3.16.9 
 - tool-ninja @ 1.10.2 
 - toolchain-esp32ulp @ 2.35.0-20220830 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Installing ESP-IDF's Python dependencies
Collecting urllib3<2
  Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Collecting cryptography<35.0.0,>=2.1.4
  Downloading cryptography-3.4.8.tar.gz (546 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /root/.platformio/penv/.espidf-4.4.5/bin/python /root/.platformio/penv/.espidf-4.4.5/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-qsvqfnm0/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --prefer-binary -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4'
       cwd: None
  Complete output (90 lines):
  Collecting setuptools>=40.6.0
    Downloading setuptools-68.2.2-py3-none-any.whl (807 kB)
  Collecting wheel
    Downloading wheel-0.41.2-py3-none-any.whl (64 kB)
  Collecting cffi>=1.12
    Downloading cffi-1.16.0.tar.gz (512 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Installing backend dependencies: started
    Installing backend dependencies: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'done'
  Collecting setuptools-rust>=0.11.4
    Downloading setuptools_rust-1.7.0-py3-none-any.whl (25 kB)
  Collecting pycparser
    Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
  Collecting typing-extensions>=3.7.4.3
    Downloading typing_extensions-4.8.0-py3-none-any.whl (31 kB)
  Collecting tomli>=1.2.1
    Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
  Collecting semantic-version<3,>=2.8.2
    Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
  Building wheels for collected packages: cffi
    Building wheel for cffi (PEP 517): started
    Building wheel for cffi (PEP 517): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /root/.platformio/penv/.espidf-4.4.5/bin/python /tmp/tmpw12evqx1_in_process.py build_wheel /tmp/tmprxjfqmon
         cwd: /tmp/pip-install-pcb64ztz/cffi_58f06e36f2b0409a80c51fcbf81dd053
    Complete output (55 lines):
    /tmp/pip-build-env-b8dl3kb7/overlay/lib/python3.9/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
    !!

            ********************************************************************************
            The license_file parameter is deprecated, use license_files instead.

            By 2023-Oct-30, you need to update your project and remove deprecated calls
            or your builds will no longer be supported.

            See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
            ********************************************************************************

    !!
      parsed = self.parsers.get(option_name, lambda x: x)(value)
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.linux-armv7l-cpython-39
    creating build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/ffiplatform.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/pkgconfig.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/__init__.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_imp_emulation.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_shimmed_dist_utils.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/recompiler.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/cparser.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/model.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/vengine_cpy.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/verifier.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/backend_ctypes.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/cffi_opcode.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/lock.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/setuptools_ext.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/api.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/error.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/commontypes.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/vengine_gen.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_cffi_include.h -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/parse_c_type.h -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_embedding.h -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_cffi_errors.h -> build/lib.linux-armv7l-cpython-39/cffi
    warning: build_py: byte-compiling is disabled, skipping.

    running build_ext
    building '_cffi_backend' extension
    creating build/temp.linux-armv7l-cpython-39
    creating build/temp.linux-armv7l-cpython-39/src
    creating build/temp.linux-armv7l-cpython-39/src/c
    arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/.platformio/penv/.espidf-4.4.5/include -I/usr/include/python3.9 -c src/c/_cffi_backend.c -o build/temp.linux-armv7l-cpython-39/src/c/_cffi_backend.o
    src/c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory

*************************************************************
* Looking for ffi.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:ffi.h"
* Web  > https://registry.platformio.org/search?q=header:ffi.h
*
*************************************************************

       15 | #include <ffi.h>
          |          ^~~~~~~
    compilation terminated.
    error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1
    ----------------------------------------
    ERROR: Failed building wheel for cffi
  Failed to build cffi
  ERROR: Could not build wheels for cffi which use PEP 517 and cannot be installed directly
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/cc/98/8a258ab4787e6f835d350639792527d2eb7946ff9fc0caca9c3f4cf5dcfe/cryptography-3.4.8.tar.gz#sha256=94cc5ed4ceaefcbe5bf38c8fba6a21fc1d365bb8fb826ea1688e3370b2e24a1c (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6). Command errored out with exit status 1: /root/.platformio/penv/.espidf-4.4.5/bin/python /root/.platformio/penv/.espidf-4.4.5/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-qsvqfnm0/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --prefer-binary -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4' Check the logs for full command output.
  Downloading cryptography-3.4.7.tar.gz (546 kB)
  Installing build dependencies: started

[tigers75]

I think it's not in 2023.10. I installed 2023.11 dev and I'm not getting this error anymore (but it still doesn't compile for a Shelly device, I have to Investigate more before opening another issue, and it's not even a beta yet)

[IGitYou]

Agreed with above. Using the Dev branch and having the same results as @tigers75 I do not compile on an ESP32. No error but it may be my container is shutting down. It runs for like an hour and then suggests retrying. My dev env (not on pi hardware) still works.

I can provide logging or whatever and although this error went away, the pursuit of completing a compile and installing is still busted in my env.

TLDR: @jesserockz merge of https://github.com/esphome/esphome/pull/5473 stopped the error from occurring but did not resume the ability to compile or install on these boards.

[myLens]

Some things become better - some worse. WhoTF has decided that we have to have cryptography, RUST and that kind of stuff ?

[IGitYou]

Latest logs from latest image shows improvement but ultimately fails in a new place now. Log excerpts from updated point of failure:

` Installing collected packages: pyparsing, urllib3, tqdm, six, pyyaml, pycparser, packaging, msgpack, kconfiglib, idna, future, filelock, contextlib2, colorama, click, charset-normalizer, certifi, schema, requests, cffi, requests-toolbelt, requests-file, cryptography, cachecontrol, idf-component-manager Running setup.py install for future: started DEPRECATION: future is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559 Running setup.py install for future: finished with status 'done' Successfully installed cachecontrol-0.13.1 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.1 click-8.1.7 colorama-0.4.6 contextlib2-21.6.0 cryptography-3.4.8 filelock-3.12.4 future-0.18.3 idf-component-manager-1.4.1 idna-3.4 kconfiglib-13.7.1 msgpack-1.0.7 packaging-23.2 pycparser-2.21 pyparsing-2.3.1 pyyaml-6.0.1 requests-2.31.0 requests-file-1.5.1 requests-toolbelt-1.0.0 schema-0.7.5 six-1.16.0 tqdm-4.66.1 urllib3-1.26.18 Reading CMake configuration... -- The C compiler identification is unknown -- The CXX compiler identification is unknown -- The ASM compiler identification is unknown -- Found assembler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- Check for working C compiler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- Check for working C compiler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- broken -- Configuring incomplete, errors occurred! See also "/config/.esphome/build/esp32-packagedpilot/.pioenvs/esp32-packagedpilot/CMakeFiles/CMakeOutput.log". See also "/config/.esphome/build/esp32-packagedpilot/.pioenvs/esp32-packagedpilot/CMakeFiles/CMakeError.log".

fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). CMake Error at /config/.esphome/platformio/packages/tool-cmake/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake:60 (message): The C compiler

"/config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: /config/.esphome/build/esp32-packagedpilot/.pioenvs/esp32-packagedpilot/CMakeFiles/CMakeTmp

Run Build Command(s):/config/.esphome/platformio/packages/tool-ninja/ninja cmTC_c6b71 && [1/2] Building C object CMakeFiles/cmTC_c6b71.dir/testCCompiler.c.obj
FAILED: CMakeFiles/cmTC_c6b71.dir/testCCompiler.c.obj 
/config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc   -mlongcalls -Wno-frame-address -o CMakeFiles/cmTC_c6b71.dir/testCCompiler.c.obj   -c testCCompiler.c
/bin/sh: 1: /config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc: not found
ninja: build stopped: subcommand failed.

CMake will not be able to correctly generate this project. Call Stack (most recent call first): /config/.esphome/platformio/packages/framework-espidf/tools/cmake/project.cmake:298 (__project) CMakeLists.txt:3 (project)

======================== [FAILED] Took 3804.95 seconds ======================== `

[iw0der]

Hi guys, on RPI3 with HA updated to the latest version, including the operating system and addons, I have this problem that blocks the compilation of the blueprint for ns-panel. Looking around github and more generally on the internet I seem to understand that it is a problem that has been known for a couple of years and is present on various systems... I tried to look for some solutions but evidently I lack the right knowledge to deal with the problem!

So in the meantime I'm listening here to see if there's anyone able to find a workaround or in any case waiting for any fix released by the developers.

Thank you for your attention and commitment!

73, Arturo.

[myLens]

Hi Arturo, I am not really an expert too but had the same problem with my HA installation. Then a guy from ESPHome forum told me it has to be a 64-Bit version of HA and mine was old and only 32Bit. maybe that helps. BR, Tom

[iw0der]

Thanks Tom @myLens, I imagine mine is also 32 bit; HA is installed on a Raspberry Pi 3 which has limited resources; but I seem to have understood that someone has the same problem also on installations on RPI4 which usually uses a 64 bit operating system.

Strange, however, that in the pull 5473 it seems that the problem has been identified and resolved some weeks ago... maybe it's just a matter of waiting for the next HA version !?!?!? Or I misunderstood and there is no solution for some installations! :-(

73, Arturo.

[myLens]

Hi Arturo, Well I don't know if there is a solution if your Pi would run on 64 Bits. But you can simply check your OS Version In Menu "Settings"-"System"-"Hardware". The member in the ESPHome Forum told me that with a 32 Bit operating system, many things like the right Phython version, Cryptography, Rust compiler etc. would not be running. Good luck, Tom

[Turkules]

I get similar errors :/

INFO ESPHome 2023.10.6
INFO Reading configuration /config/esp32-bluetooth-proxy-d78d6c.yaml...
INFO Generating C++ source...
INFO Compiling app...
Processing esp32-bluetooth-proxy-d78d6c (board: esp32dev; framework: espidf; platform: platformio/espressif32@5.4.0)
--------------------------------------------------------------------------------
Platform Manager: Installing platformio/espressif32 @ 5.4.0
INFO Installing platformio/espressif32 @ 5.4.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Platform Manager: espressif32@5.4.0 has been installed!
INFO espressif32@5.4.0 has been installed!
Tool Manager: Installing espressif/toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
INFO Installing espressif/toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%          
Tool Manager: toolchain-xtensa-esp32@8.4.0+2021r2-patch5 has been installed!
INFO toolchain-xtensa-esp32@8.4.0+2021r2-patch5 has been installed!
Tool Manager: Installing espressif/toolchain-esp32ulp @ 2.35.0-20220830
INFO Installing espressif/toolchain-esp32ulp @ 2.35.0-20220830
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: toolchain-esp32ulp@2.35.0-20220830 has been installed!
INFO toolchain-esp32ulp@2.35.0-20220830 has been installed!
Tool Manager: Installing platformio/framework-espidf @ ~3.40405.0
INFO Installing platformio/framework-espidf @ ~3.40405.0
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%          
Tool Manager: framework-espidf@3.40405.230623 has been installed!
INFO framework-espidf@3.40405.230623 has been installed!
Tool Manager: Installing platformio/tool-esptoolpy @ ~1.40400.0
INFO Installing platformio/tool-esptoolpy @ ~1.40400.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-esptoolpy@1.40400.0 has been installed!
INFO tool-esptoolpy@1.40400.0 has been installed!
Tool Manager: Installing platformio/tool-cmake @ ~3.16.0
INFO Installing platformio/tool-cmake @ ~3.16.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%          
Tool Manager: tool-cmake@3.16.9 has been installed!
INFO tool-cmake@3.16.9 has been installed!
Tool Manager: Installing platformio/tool-ninja @ ^1.7.0
INFO Installing platformio/tool-ninja @ ^1.7.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-ninja@1.10.2 has been installed!
INFO tool-ninja@1.10.2 has been installed!
Tool Manager: Installing platformio/tool-scons @ ~4.40502.0
INFO Installing platformio/tool-scons @ ~4.40502.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: tool-scons@4.40502.0 has been installed!
INFO tool-scons@4.40502.0 has been installed!
Library Manager: Installing esphome/noise-c @ 0.1.4
INFO Installing esphome/noise-c @ 0.1.4
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Library Manager: noise-c@0.1.4 has been installed!
INFO noise-c@0.1.4 has been installed!
Library Manager: Resolving dependencies...
INFO Resolving dependencies...
Library Manager: Installing esphome/libsodium @ 1.10018.1
INFO Installing esphome/libsodium @ 1.10018.1
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Library Manager: libsodium@1.10018.1 has been installed!
INFO libsodium@1.10018.1 has been installed!
Library Manager: Installing esphome/Improv @ 1.2.3
INFO Installing esphome/Improv @ 1.2.3
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Library Manager: Improv@1.2.3 has been installed!
INFO Improv@1.2.3 has been installed!
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - framework-espidf @ 3.40405.230623 (4.4.5) 
 - tool-cmake @ 3.16.9 
 - tool-ninja @ 1.10.2 
 - toolchain-esp32ulp @ 2.35.0-20220830 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Creating a new virtual environment for IDF Python dependencies
Installing ESP-IDF's Python dependencies
Collecting urllib3<2
  Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
Collecting cryptography<35.0.0,>=2.1.4
  Downloading cryptography-3.4.8.tar.gz (546 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /root/.platformio/penv/.espidf-4.4.5/bin/python /root/.platformio/penv/.espidf-4.4.5/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-rr1f8okw/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4'
       cwd: None
  Complete output (86 lines):
  Collecting setuptools>=40.6.0
    Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
  Collecting wheel
    Downloading wheel-0.41.3-py3-none-any.whl (65 kB)
  Collecting cffi>=1.12
    Downloading cffi-1.16.0.tar.gz (512 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Installing backend dependencies: started
    Installing backend dependencies: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'done'
  Collecting setuptools-rust>=0.11.4
    Downloading setuptools_rust-1.8.1-py3-none-any.whl (26 kB)
  Collecting pycparser
    Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
  Collecting semantic-version<3,>=2.8.2
    Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
  Collecting tomli>=1.2.1
    Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
  Building wheels for collected packages: cffi
    Building wheel for cffi (PEP 517): started
    Building wheel for cffi (PEP 517): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /root/.platformio/penv/.espidf-4.4.5/bin/python /tmp/tmpgjcpcnar_in_process.py build_wheel /tmp/tmpjt1tmoxt
         cwd: /tmp/pip-install-n4sa9qqp/cffi_dfa63e3e4f2f45a282ed80bd89ab1264
    Complete output (53 lines):
    /tmp/pip-build-env-jr6xz_87/overlay/lib/python3.9/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
    !!

            ********************************************************************************
            The license_file parameter is deprecated, use license_files instead.

            This deprecation is overdue, please update your project and remove deprecated
            calls to avoid build errors in the future.

            See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
            ********************************************************************************

    !!
      parsed = self.parsers.get(option_name, lambda x: x)(value)
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.linux-armv7l-cpython-39
    creating build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/cffi_opcode.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/api.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/__init__.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/verifier.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/ffiplatform.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/setuptools_ext.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_imp_emulation.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/commontypes.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/lock.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/pkgconfig.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/vengine_cpy.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/cparser.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_shimmed_dist_utils.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/model.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/recompiler.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/backend_ctypes.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/error.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/vengine_gen.py -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_cffi_include.h -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/parse_c_type.h -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_embedding.h -> build/lib.linux-armv7l-cpython-39/cffi
    copying src/cffi/_cffi_errors.h -> build/lib.linux-armv7l-cpython-39/cffi
    running build_ext
    building '_cffi_backend' extension
    creating build/temp.linux-armv7l-cpython-39
    creating build/temp.linux-armv7l-cpython-39/src
    creating build/temp.linux-armv7l-cpython-39/src/c
    arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/.platformio/penv/.espidf-4.4.5/include -I/usr/include/python3.9 -c src/c/_cffi_backend.c -o build/temp.linux-armv7l-cpython-39/src/c/_cffi_backend.o
    src/c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory

*************************************************************
* Looking for ffi.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:ffi.h"
* Web  > https://registry.platformio.org/search?q=header:ffi.h
*
*************************************************************

       15 | #include <ffi.h>
          |          ^~~~~~~
    compilation terminated.
    error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1
    ----------------------------------------
    ERROR: Failed building wheel for cffi
  Failed to build cffi
  ERROR: Could not build wheels for cffi which use PEP 517 and cannot be installed directly
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/cc/98/8a258ab4787e6f835d350639792527d2eb7946ff9fc0caca9c3f4cf5dcfe/cryptography-3.4.8.tar.gz#sha256=94cc5ed4ceaefcbe5bf38c8fba6a21fc1d365bb8fb826ea1688e3370b2e24a1c (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6). Command errored out with exit status 1: /root/.platformio/penv/.espidf-4.4.5/bin/python /root/.platformio/penv/.espidf-4.4.5/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-rr1f8okw/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"'' 'setuptools-rust>=0.11.4' Check the logs for full command output.
  Downloading cryptography-3.4.7.tar.gz (546 kB)

[ssieb]

@Turkules you're running an old esphome version. This issue was fixed after that.

[rvanderven]

2023.10.6 is still the released one, the rest is dev

[tigers75]

To be honest this issue (that is related to wrong dependencies in the docker image is still present in latest STABLE docker image of ESPHome (2023.10.6) which does not work on arm processor (works fine on windows docker). The latest BETA docker image (2023.11.0b3) still fails on arm, with a different error: It stucks forever with this message: Collecting certifi>=2017.4.17 Downloading certifi-2023.7.22-py3-none-any.whl (158 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 2.0 MB/s eta 0:00:00 Building wheels for collected packages: cryptography, pyyaml, msgpack Building wheel for cryptography (pyproject.toml): started Building wheel for cryptography (pyproject.toml): still running...

I didn't bother trying the dev branch, I'll just use windows docker when I need it until it's all been sorted.

EDIT: that's the full message after 1650 seconds of building the firmware:

`Successfully installed cachecontrol-0.13.1 certifi-2023.7.22 cffi-1.16.0 charset-normalizer-3.3.2 click-8.1.7 colorama-0.4.6 contextlib2-21.6.0 cryptography-3.4.8 filelock-3.13.1 future-0.18.3 idf-component-manager-1.4.1 idna-3.4 kconfiglib-13.7.1 msgpack-1.0.7 packaging-23.2 pycparser-2.21 pyparsing-2.3.1 pyyaml-6.0.1 requests-2.31.0 requests-file-1.5.1 requests-toolbelt-1.0.0 schema-0.7.5 six-1.16.0 tqdm-4.66.1 urllib3-1.26.18 Reading CMake configuration... -- Found Git: /usr/bin/git (found version "2.39.2") -- The C compiler identification is unknown -- The CXX compiler identification is unknown -- The ASM compiler identification is unknown -- Found assembler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- Check for working C compiler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- Check for working C compiler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc -- broken -- Configuring incomplete, errors occurred! See also "/config/.esphome/build/shelly-plus-1/.pioenvs/shelly-plus-1/CMakeFiles/CMakeOutput.log". See also "/config/.esphome/build/shelly-plus-1/.pioenvs/shelly-plus-1/CMakeFiles/CMakeError.log".

fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). CMake Error at /config/.esphome/platformio/packages/tool-cmake/share/cmake-3.16/Modules/CMakeTestCCompiler.cmake:60 (message): The C compiler

"/config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: /config/.esphome/build/shelly-plus-1/.pioenvs/shelly-plus-1/CMakeFiles/CMakeTmp

Run Build Command(s):/config/.esphome/platformio/packages/tool-ninja/ninja cmTC_4acc7 && [1/2] Building C object CMakeFiles/cmTC_4acc7.dir/testCCompiler.c.obj
FAILED: CMakeFiles/cmTC_4acc7.dir/testCCompiler.c.obj 
/config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc   -mlongcalls -Wno-frame-address -o CMakeFiles/cmTC_4acc7.dir/testCCompiler.c.obj   -c testCCompiler.c
/bin/sh: 1: /config/.esphome/platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-gcc: not found
ninja: build stopped: subcommand failed.

CMake will not be able to correctly generate this project. Call Stack (most recent call first): /config/.esphome/platformio/packages/framework-espidf/tools/cmake/project.cmake:298 (__project) CMakeLists.txt:3 (project)

======================== [FAILED] Took 1650.85 seconds ========================`

[ssieb]

2023.10.6 is still the released one, the rest is dev

Somehow I misread that version number. Anyway, I see this in the logs: /root/.platformio/penv/.espidf-4.4.5/lib/python3.9/site-packages/pip install 3.9 is too old for esphome.

[ssieb]

@tigers75 that's a different issue, which is most likely because you have a 32-bit userspace on a 64-bit kernel.