Installation failed, unclear why (FAILED: python/gen/matter_build_venv/compiled_requirements.txt) (CON-1215)

[JetForMe]

Describe the bug I'm running through the installation instructions for Matter support, and ran into an error I don't understand. You can see the lengthy transcript here. Excerpt:

[511/1099] python3 ../../third_party/pigweed/repo/pw_build/py/pw_build/python_runner.py --gn-root ../../ --current-path ../.. --default-toolchain=//build/toolchain/host:mac_arm64_gcc --current-toolchain=//third_party/pigweed/repo/pw_build/python_toolchain:python --capture-output --module piptools --python-virtualenv-config python/gen/matter_build_venv/venv_metadata.json --python-dep-list-files python/gen/matter_build_venv._compile_requirements_metadata_path_list.txt -- compile --resolver=backtracking --allow-unsafe --output-file python/gen/matter_build_venv/compiled_requirements.txt python/gen/matter_build_venv/generated_requirements.txt ../../scripts/setup/requirements.build.txt
FAILED: python/gen/matter_build_venv/compiled_requirements.txt 
python3 ../../third_party/pigweed/repo/pw_build/py/pw_build/python_runner.py --gn-root ../../ --current-path ../.. --default-toolchain=//build/toolchain/host:mac_arm64_gcc --current-toolchain=//third_party/pigweed/repo/pw_build/python_toolchain:python --capture-output --module piptools --python-virtualenv-config python/gen/matter_build_venv/venv_metadata.json --python-dep-list-files python/gen/matter_build_venv._compile_requirements_metadata_path_list.txt -- compile --resolver=backtracking --allow-unsafe --output-file python/gen/matter_build_venv/compiled_requirements.txt python/gen/matter_build_venv/generated_requirements.txt ../../scripts/setup/requirements.build.txt
    error: subprocess-exited-with-error

    × Getting requirements to build wheel did not run successfully.

Environment

• ESP-Matter Commit Id: d83caf7fefea
• ESP-IDF Commit Id: v5.2.2
• SoC (eg: ESP32 or ESP32-C3):
• Device Logs (Please attach the log file):
• Host Machine OS: macOS 14.5, Xcode 16beta1
• Commissioner app and versions if present:
• Commissioner's logs if present:

Any additional details ...

[9zigen]

I have the same problem. After downgrading Python version from 3.12 to 3.9 it works.

[ntwyman]

I also had to downgrade python 3.12 to 3.9.2 to get past issues installing esp-matter on an M1 Mac.

[farhan-syed]

Also would like to add incase someone sees this and still is getting the issue even though you have 3.9. May need to install Rosetta 2 if you don't already have it. Installing Rosetta fixed it for me.

[shubhamdp]

@JetForMe can you please provide the complete failure logs and what python version are you using?

[JetForMe]

@JetForMe can you please provide the complete failure logs and what python version are you using?

Oh I'm sorry, I didn't realize I had the wrong link in there. I've edited the original comment, but you can find it here: https://gist.github.com/JetForMe/f410ac9057d2a6c0990f2f53c04c6915

% python --version
Python 3.12.3

[retsyx]

Espressif, please fix this. Having to downgrade python to 3.9 is not a solution.

Edit: this seems to be an old known issue in the upstream matter library. Which, in turn, is because of a breaking change in Python 3.11.

In the meantime, this awful workaround in esp-idf prioritizes using older versions of installed python so that it can work.

diff --git a/tools/detect_python.sh b/tools/detect_python.sh
index bc3e65c59a..f98ab37bac 100644
--- a/tools/detect_python.sh
+++ b/tools/detect_python.sh
@@ -12,7 +12,7 @@ OLDEST_PYTHON_SUPPORTED_MINOR=8

 ESP_PYTHON=python

-for p_cmd in python3 python python3.8 python3.9 python3.10 python3.11 python3.12; do
+for p_cmd in python3.8 python3.9 python3.10 python3.11 python3.12 python3; do
     $p_cmd --version >/dev/null 2>&1 || continue
     echo "Checking \"$p_cmd\" ..."

[sldr]

Another workaround that worked for me is:

echo "cython<3" > /tmp/constraint.txt
export PIP_CONSTRAINT=/tmp/constraint.txt
cd esp-matter
./install.sh

Note:

• This seems to get you down the road to another module using imp instead of importlib. I think egg or maybe python has made me hangry.
• Credit for find is really in pyyaml issue commant

Laters, SLDR (Stephen L. De Rudder)

IT WORKED!!! I was able to build the esp-matter/examples/managed_component_light demo and flash it to an esp32h2 and add it to my homeassistant yellow matter over thread network. I am running Ubuntu 24.04 with Python 3.12.3. The esp-idf git clone was switched to 5.2.1 (per doc) and esp-matter is on branch main (specifically 794f30e4f04166614588e2fa21cfaf085cf1cf96). I just ignored the other (down the road) error mentioned in previous note. I used the code from here instead of the QR code pic but I expect the pic would have worked. If I missed anything let me know. My plan is to create a sensor (a bunch on a single esp32h2 for my home alarm system).