mesonbuild / meson

The Meson Build System
http://mesonbuild.com
Apache License 2.0
5.59k stars 1.62k forks source link

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 16800744: invalid start byte #13443

Open butterflyhigh opened 3 months ago

butterflyhigh commented 3 months ago

Describe the bug When I try to run meson setup while chrooted into a minimal Linux system, it fails due to the unicode error in the title. It does not fail on the host system.

The output of meson setup from chroot:

The Meson build system
Version: 1.5.0
Source dir: /home/portage
Build dir: /home/portage/build
Build type: native build
Project name: portage
Project version: 3.0.65
C compiler for the host machine: clang (clang 18.1.8 "clang version 18.1.8")
C linker for the host machine: clang ld.bfd 2.42.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (/usr/bin/python)
Program sed found: YES (/usr/bin/sed)
Configuring const.py using configuration
Configuring installation.py using configuration
Configuring __init__.py using configuration
Configuring AbstractEbuildProcess.py using configuration
Configuring create_depgraph_params.py using configuration
Found pkg-config: YES (/usr/bin/pkg-config) 2.1.1
Run-time dependency python found: YES 3.12
Configuring METADATA using configuration
Program patch found: YES (/usr/bin/patch)
Configuring make.conf.example with command
Program sphinx-apidoc found: NO
Program sphinx-build found: NO
Configuring version using configuration
Program xmlto found: NO
Configuring color.map.5 using configuration
Configuring dispatch-conf.1 using configuration
Configuring ebuild.1 using configuration
Configuring ebuild.5 using configuration
Configuring egencache.1 using configuration
Configuring emaint.1 using configuration
Configuring emerge.1 using configuration
Configuring emirrordist.1 using configuration
Configuring env-update.1 using configuration
Configuring etc-update.1 using configuration
Configuring fixpackages.1 using configuration
Configuring glsa-check.1 using configuration
Configuring make.conf.5 using configuration
Configuring portage.5 using configuration
Configuring quickpkg.1 using configuration
Configuring xpak.5 using configuration
Configuring color.map.5 using configuration
Configuring dispatch-conf.1 using configuration
Configuring ebuild.1 using configuration
Configuring env-update.1 using configuration
Configuring etc-update.1 using configuration
Configuring fixpackages.1 using configuration
Build targets in project: 1

Found ninja-1.12.1 at /usr/bin/ninja
Traceback (most recent call last):                                                                                                                                                                                
  File "/usr/lib/python3.12/site-packages/mesonbuild/mesonmain.py", line 188, in run
    return options.run_func(options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mesonbuild/msetup.py", line 364, in run
    app.generate()
  File "/usr/lib/python3.12/site-packages/mesonbuild/msetup.py", line 187, in generate
    return self._generate(env, capture, vslite_ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mesonbuild/msetup.py", line 252, in _generate
    captured_compile_args = intr.backend.generate(capture, vslite_ctx)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mesonbuild/backend/ninjabackend.py", line 648, in generate
    self.generate_target(t)
  File "/usr/lib/python3.12/site-packages/mesonbuild/backend/ninjabackend.py", line 1048, in generate_target
    o, s = self.generate_single_compile(target, src, False, [],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mesonbuild/backend/ninjabackend.py", line 2984, in generate_single_compile
    self.create_target_source_introspection(target, compiler, commands, [src], [], unity_sources)
  File "/usr/lib/python3.12/site-packages/mesonbuild/backend/ninjabackend.py", line 822, in create_target_source_introspection
    parameters = parameters.to_native(copy=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mesonbuild/compilers/mixins/clike.py", line 92, in to_native
    default_dirs = self.compiler.get_default_include_dirs()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mesonbuild/compilers/mixins/gnu.py", line 402, in get_default_include_dirs
    return gnulike_default_include_dirs(tuple(self.get_exelist(ccache=False)), self.language).copy()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mesonbuild/compilers/mixins/gnu.py", line 327, in gnulike_default_include_dirs
    _, stdout, _ = mesonlib.Popen_safe(cmd, stderr=subprocess.STDOUT, env=env)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/mesonbuild/utils/universal.py", line 1554, in Popen_safe
    o, e = p.communicate(write)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 1196, in communicate
    stdout = self.stdout.read()
             ^^^^^^^^^^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 16800744: invalid start byte

ERROR: Unhandled python exception

    This is a Meson bug and should be reported!

The output of meson setup from the host system (Arch Linux):

The Meson build system
Version: 1.5.0
Source dir: /home/coolkid/portage
Build dir: /home/coolkid/portage/build
Build type: native build
Project name: portage
Project version: 3.0.65
C compiler for the host machine: ccache cc (gcc 14.1.1 "cc (GCC) 14.1.1 20240522")
C linker for the host machine: cc ld.bfd 2.42.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (/usr/bin/python)
Program sed found: YES (/usr/bin/sed)
Configuring const.py using configuration
Configuring installation.py using configuration
Configuring __init__.py using configuration
Configuring AbstractEbuildProcess.py using configuration
Configuring create_depgraph_params.py using configuration
Found pkg-config: YES (/usr/bin/pkg-config) 2.1.1
Run-time dependency python found: YES 3.12
Configuring METADATA using configuration
Program patch found: YES (/usr/bin/patch)
Configuring make.conf.example with command
Program sphinx-apidoc found: NO
Program sphinx-build found: NO
Configuring version using configuration
Program xmlto found: NO
Configuring color.map.5 using configuration
Configuring dispatch-conf.1 using configuration
Configuring ebuild.1 using configuration
Configuring ebuild.5 using configuration
Configuring egencache.1 using configuration
Configuring emaint.1 using configuration
Configuring emerge.1 using configuration
Configuring emirrordist.1 using configuration
Configuring env-update.1 using configuration
Configuring etc-update.1 using configuration
Configuring fixpackages.1 using configuration
Configuring glsa-check.1 using configuration
Configuring make.conf.5 using configuration
Configuring portage.5 using configuration
Configuring quickpkg.1 using configuration
Configuring xpak.5 using configuration
Configuring color.map.5 using configuration
Configuring dispatch-conf.1 using configuration
Configuring ebuild.1 using configuration
Configuring env-update.1 using configuration
Configuring etc-update.1 using configuration
Configuring fixpackages.1 using configuration
Build targets in project: 1

Found ninja-1.12.1 at /usr/bin/ninja

To Reproduce The repository being built

Expected behavior I expected it to build successfully.

system parameters

eli-schwartz commented 3 months ago

The error message comes from here: https://github.com/mesonbuild/meson/blob/864919973f73ec2c6ae6ca01be7fe5e8b950df12/mesonbuild/compilers/mixins/gnu.py#L320-L327

Inside the chroot, you have clang 18, and it attempts to:

This apparently fails because the combination of stdout and stderr for clang when told to use the C locale, is... not utf-8? This does not seem like a thing that should happen.