Can't build on Ubuntu 24.04 LTS Desktop (CON-1143)

PichuChen commented 4 months ago

Describe the bug A clear and concise description of what the bug is.

Environment

ESP-Matter Commit Id: 89f47b6c8a3c4f9743d5e713536b3a23d60e8af5
SoC (eg: ESP32 or ESP32-C3): N/A

Host Machine OS: Ubuntu 24.04LTS

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04 LTS
Release:        24.04
Codename:       noble

python: Python 3.12.3

$ ninja --version 1.11.1

Log:

Downloading and installing packages into local source directory:

  Setting up CIPD package manager...done (2.6s)
  Setting up Project actions........skipped (0.1s)
  Setting up Python environment.....[-]
ninja: Entering directory `/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out'
[1/589] python3 ../../third_party/pigweed/repo/pw_build/py/pw_build/python_runner.py --gn-root ../../ --current-path ../.. --default-toolchain=//build/toolchain/linux:linux_x64_gcc --current-toolchain=//third_party/pigweed/repo/pw_build/python_toolchain:python --capture-output --module piptools --python-virtualenv-config python/gen/matter_build_venv/venv_metadata.json --python-dep-list-files python/gen/matter_build_venv._compile_requirements_metadata_path_list.txt -- compile --resolver=backtracking --allow-unsafe --output-file python/gen/matter_build_venv/compiled_requirements.txt python/gen/matter_build_venv/generated_requirements.txt ../../scripts/setup/requirements.build.txt
FAILED: python/gen/matter_build_venv/compiled_requirements.txt
python3 ../../third_party/pigweed/repo/pw_build/py/pw_build/python_runner.py --gn-root ../../ --current-path ../.. --default-toolchain=//build/toolchain/linux:linux_x64_gcc --current-toolchain=//third_party/pigweed/repo/pw_build/python_toolchain:python --capture-output --module piptools --python-virtualenv-config python/gen/matter_build_venv/venv_metadata.json --python-dep-list-files python/gen/matter_build_venv._compile_requirements_metadata_path_list.txt -- compile --resolver=backtracking --allow-unsafe --output-file python/gen/matter_build_venv/compiled_requirements.txt python/gen/matter_build_venv/generated_requirements.txt ../../scripts/setup/requirements.build.txt
    error: subprocess-exited-with-error

    × Getting requirements to build wheel did not run successfully.
    │ exit code: 1
    ╰─> [54 lines of output]
        running egg_info
        writing lib/PyYAML.egg-info/PKG-INFO
        writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
        writing top-level names to lib/PyYAML.egg-info/top_level.txt
        Traceback (most recent call last):
          File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
            main()
          File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
            json_out['return_val'] = hook(**hook_input['kwargs'])
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
            return hook(config_settings)
                   ^^^^^^^^^^^^^^^^^^^^^
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
            return self._get_build_requires(config_settings, requirements=['wheel'])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
            self.run_setup()
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
            exec(code, locals())
          File "<string>", line 288, in <module>
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 104, in setup
            return distutils.core.setup(**attrs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
            return run_commands(dist)
                   ^^^^^^^^^^^^^^^^^^
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
            dist.run_commands()
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
            self.run_command(cmd)
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
            super().run_command(command)
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
            cmd_obj.run()
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
            self.find_sources()
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
            mm.run()
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 550, in run
            self.add_defaults()
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 588, in add_defaults
            sdist.add_defaults(self)
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 102, in add_defaults
            super().add_defaults()
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 250, in add_defaults
            self._add_defaults_ext()
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 335, in _add_defaults_ext
            self.filelist.extend(build_ext.get_source_files())
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "<string>", line 204, in get_source_files
          File "/tmp/pip-build-env-bcem_9zo/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
            raise AttributeError(attr)
        AttributeError: cython_sources
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/piptools/__main__.py", line 19, in <module>
    cli()
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/piptools/scripts/compile.py", line 659, in cli
    results = resolver.resolve(max_rounds=max_rounds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/piptools/resolver.py", line 604, in resolve
    is_resolved = self._do_resolve(
                  ^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/piptools/resolver.py", line 636, in _do_resolve
    resolver.resolve(
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
           ^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
                ^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 293, in __init__
    super().__init__(
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
                ^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
    dist = self._prepare_distribution()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 538, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 653, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 48, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 118, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 95, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/utils/misc.py", line 697, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 311, in _call_hook
    self._subprocess_runner(
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/utils/subprocess.py", line 252, in runner
    call_subprocess(
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out/python-venv/lib/python3.12/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
pip._internal.exceptions.InstallationSubprocessError: Getting requirements to build wheel exited with 1
ninja: build stopped: subcommand failed.
['ninja', '-C', '/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out', '-v', ':python_packages.install']

Traceback (most recent call last):
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 421, in install_packages
    subprocess.check_call(ninja_cmd, stdout=outs, stderr=outs)
  File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ninja', '-C', '/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out', '-v', ':python_packages.install']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 995, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 987, in main
    return EnvSetup(**vars(parse())).setup()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 577, in setup
    result = step(spin)
             ^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 766, in virtualenv
    if not virtualenv_setup.install(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 443, in install
    install_packages(gn_target)
  File "/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 424, in install_packages
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['ninja', '-C', '/home/ubuntu/esp-matter/connectedhomeip/connectedhomeip/.environment/gn_out', '-v', ':python_packages.install']' returned non-zero exit status 1.

Any additional details ...

PichuChen commented 4 months ago

I have reinstall the os with Ubuntu 22.04 LTS Desktop, and complete the matter install.

jonsmirl commented 4 months ago

I am running 22.04LTS too and the first time you ran that was something newer (24.04?) than 22.04. 22.04 is still on python 3.10

jonsmirl@ares:~$ python --version
Python 3.10.12
jonsmirl@ares:~$ ninja --version
1.10.1

shubhamdp commented 4 months ago

@PichuChen right now the setup is only tested on Ubuntu 22.04 LTS and it may take some time to test it on 24.04 LTS. https://docs.espressif.com/projects/esp-matter/en/latest/esp32/developing.html#host-setup

PichuChen commented 4 months ago

you ran that was something newer

Yes, I known 24.04 didn't tested right now, but I didn't found the ticket related on it, so I open a ticket for this issue.

dhrishi commented 3 months ago

Even the upstream connectedhomeip does not mention 24.04 in the tested environment - https://github.com/project-chip/connectedhomeip/blob/master/docs/guides/BUILDING.md#tested-operating-systems

baylf2000 commented 1 month ago

Is there any update on this at all? Has anyone managed to make the required changes to make 24.04 work? Ubuntu 22.04 is several years old now, and 24.04 has many improvements.

KUAPT commented 4 weeks ago

Is there any new progress on this issue now? I also encountered the same problem with ubuntu24.04-LTS and encountered the same error message

KUAPT commented 3 weeks ago

python --version 3.11.x ninja --version 1.10.1 After testing, reverting to the above version can pass I guess the problem may occur in the Python version

baylf2000 commented 3 weeks ago

@KUAPT I opened an issue regarding this over at the matter project. Testing with Fedora 40 shows no issues, and that uses Python 3.12, so I don't think the issue is with Python. You can view the details here: https://github.com/project-chip/connectedhomeip/issues/34432

jonsmirl commented 3 weeks ago

It appears to be missing a python package. Fedora may have added that package and Ubuntu didn't. Maybe you can identify what is missing and manually add it.

espressif / esp-matter

Can't build on Ubuntu 24.04 LTS Desktop (CON-1143) #921