[BUG] Raspberry Pi 5 error setting up environment <jemalloc>: Unsupported system page size when setting Python environment

[jvmahon]

Reproduction steps

This is unique to Raspberry Pi 5.

This is both a problem report and a workaround, but it took quite a while to find a workaround, so if it is possible to have this fixed so the workaround isn't needed, that may help others.

I'll start with a workaround so it doesn't get lost in the text below.

To avoid a jemalloc error on RPi 5 when setting up the Matter build environment, alter the file /boot/firmware/config.txt, to add the lines:

#  Start kernel with 4K page size instead of 16k
kernel=kernel8.img

Now here's how the underlying issue is encountered:

On a Raspberry Pi5, with latest Raspian OS / all updates installed, when creating the build environment per these instructions: https://github.com/project-chip/connectedhomeip/blob/master/docs/guides/BUILDING.md#prerequisites , the step:

source scripts/bootstrap.sh

results in the error:

Downloading and installing packages into local source directory:

  Setting up CIPD package manager...done (4.9s)
  Setting up Project actions........skipped (0.1s)
  Setting up Python environment.....[-]
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
['gn', 'gen', '/home/jvm33/connectedhomeip/.environment/gn_out', '--args=chip_crypto="boringssl" dir_pigweed="/home/jvm33/connectedhomeip/third_party/pigweed/repo"']

Traceback (most recent call last):
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 402, in install_packages
    subprocess.check_call(
  File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gn', 'gen', '/home/jvm33/connectedhomeip/.environment/gn_out', '--args=chip_crypto="boringssl" dir_pigweed="/home/jvm33/connectedhomeip/third_party/pigweed/repo"']' died with <Signals.SIGABRT: 6>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 995, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 987, in main
    return EnvSetup(**vars(parse())).setup()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 577, in setup
    result = step(spin)
             ^^^^^^^^^^
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/env_setup.py", line 766, in virtualenv
    if not virtualenv_setup.install(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 443, in install
    install_packages(gn_target)
  File "/home/jvm33/connectedhomeip/third_party/pigweed/repo/pw_env_setup/py/pw_env_setup/virtualenv_setup/install.py", line 410, in install_packages
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['gn', 'gen', '/home/jvm33/connectedhomeip/.environment/gn_out', '--args=chip_crypto="boringssl" dir_pigweed="/home/jvm33/connectedhomeip/third_party/pigweed/repo"']' died with <Signals.SIGABRT: 6>.
Installing pip requirements for all...
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Bug prevalence

When activating the bootstrap script on RPi 5.

GitHub hash of the SDK that was being used

latest Matter SDK as of Jan. 12, 2024

Platform

raspi

Platform Version(s)

RPi 5

Anything else?

No response

[GenerallyClumsy]

Has annyone fixed this?

[andy31415]

Boostrap should try to create a separate virtualenv (in .virtualenv if PW_ENVIRONMENT_ROOT is not set) and use that.

The error you have seems to say "you should not install system python packages and instead use the system package manager like dpkg or similar" which is true, however bootstrap should have created a virtualenv for that. Guessing that maybe virtualenv package is not available?

Will need to debug this a bit, however for starters does this do anything for you?

virtualenv ~/pyenv

This should generate a virtual environment in ~/pyenv. If this does not work, probably a python virtualenv package is missing and maybe that is the error.

if it does work, you can try to:

source ~/pyenv/bin/activate.sh
source boostrap.sh

and then pip installs should happen in pyenv instead of systemenv. ... however this is NOT a correct workaround: bootstrap should create its own virtualenv, so we have to figure out why it does not.

[jvmahon]

The underlying issue is with jemalloc being set for an older pagesizes. There's some discussion of that here: https://github.com/jemalloc/jemalloc/issues/2639

It looks like jemalloc has not been updated in a while (last release 5.3.0 was May 2022), so at present, the workaround in my original post may be the easiest workaround.

[fishandphil]

Good morning

I am encountering the very same error on Raspberry Pi 5..

However @jvmahon , I am unable to find the file "/boot/firmware/config.txt" Could you give a little bit more detail on where to find it? Is it mean to be from the root of connectedhomeip/boot/firmware/config.txt ?

Thanks a lot..

[jvmahon]

However @jvmahon , I am unable to find the file "/boot/firmware/config.txt"

I think /boot/firmware/config.txt might be link to /boot/config.txt Can you find a file /boot/config.txt? Try editing that instead.

Its been a long time since I originally posted this and am no longer using RPi 5 for a development environment, so I can't even check what I did right now (sorry). You should be able to google for jemalloc error kernel8.img and find others who have solve this (that's how I originally found a solution).