search

mesonbuild / meson

The Meson Build System
http://mesonbuild.com
Apache License 2.0
5.33k stars 1.53k forks source link

Build fails if loads polly with `-Xclang -load -Xclang LLVMPolly.so` #11184

Open skbeh opened 1 year ago

skbeh commented 1 year ago

When building with CFLAGS set to -Xclang -load -Xclang LLVMPolly.so, meson incorrectly inserts -Wl,--start-group and -Wl,--end-group in the compile command. (etc. -Xclang -Wl,--start-group -load -Xclang LLVMPolly.so -Wl,--end-group, sometimes -Xclang -Wl,--start-group -load -Xclang LLVMPolly.so (some parameters) -Wl,--end-group)

When I remove https://github.com/mesonbuild/meson/blob/e4bbc630b67ef97ad842badd00855e64cff12e13/mesonbuild/compilers/mixins/clike.py#L87-L90, the problem disappears.

aleden commented 1 year ago

This is a big problem for me. I'm configuring QEMU like this:

./configure --extra-cflags="-Xclang -load -Xclang /usr/local/lib/carbon-collect.so -Xclang -add-plugin -Xclang carbon-collect -Xclang -plugin-arg-carbon-collect -Xclang $(pwd) -Xclang -plugin-arg-carbon-collect -Xclang $(pwd)"

Meson inserts "-Wl,--start-group" after the first "-Xclang", and consequently clang "interprets" /usr/local/lib/carbon-collect.so as an input file (of some sort?) and the message I see from meson is...

  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line 194, in run
    return options.run_func(options)
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 307, in run
    app.generate()
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 186, in generate
    self._generate(env)
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 230, in _generate
    intr.run()
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreter/interpreter.py", line 2895, in run
    super().run()
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 149, in run
    self.evaluate_codeblock(self.ast, start=1)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 174, in evaluate_codeblock
    raise e
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 167, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_statement
    return self.evaluate_if(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 276, in evaluate_if
    self.evaluate_codeblock(i.block)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 174, in evaluate_codeblock
    raise e
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 167, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_statement
    return self.evaluate_if(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 263, in evaluate_if
    result = self.evaluate_statement(i.condition)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 184, in evaluate_statement
    return self.method_call(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/interpreterbase.py", line 491, in method_call
    res = obj.method_call(method_name, args, kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/baseobjects.py", line 92, in method_call
    return method(args, kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/decorators.py", line 277, in wrapper
    return f(*nargs, **wrapped_kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase/decorators.py", line 581, in wrapper
    return f(*wrapped_args, **wrapped_kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreter/compiler.py", line 535, in has_header_method
    return self._has_header_impl(args[0], kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreter/compiler.py", line 520, in _has_header_impl
    haz, cached = self.compiler.has_header(hname, kwargs['prefix'], self.environment,
  File "/usr/lib/python3/dist-packages/mesonbuild/compilers/mixins/clike.py", line 366, in has_header
    return self.compiles(code, env, extra_args=extra_args,
  File "/usr/lib/python3/dist-packages/mesonbuild/compilers/compilers.py", line 1259, in compiles
    with self._build_wrapper(code, env, extra_args, dependencies, mode, disable_cache=disable_cache) as p:
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/lib/python3/dist-packages/mesonbuild/compilers/compilers.py", line 1251, in _build_wrapper
    with self.cached_compile(code, env.coredata, extra_args=args, mode=mode, temp_dir=env.scratch_dir) as r:
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/lib/python3/dist-packages/mesonbuild/compilers/compilers.py", line 859, in cached_compile
    with self.compile(code, extra_args=extra_args, mode=mode, want_output=False, temp_dir=temp_dir) as p:
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/lib/python3/dist-packages/mesonbuild/compilers/compilers.py", line 828, in compile
    p, stdo, stde = Popen_safe(command_list, cwd=tmpdirname, env=os_env)
  File "/usr/lib/python3/dist-packages/mesonbuild/utils/universal.py", line 1421, in Popen_safe
    o, e = p.communicate(write)
  File "/usr/lib/python3.10/subprocess.py", line 1154, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.10/subprocess.py", line 2043, in _communicate
    stdout = self._translate_newlines(stdout,
  File "/usr/lib/python3.10/subprocess.py", line 1031, in _translate_newlines
    data = data.decode(encoding, errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc8 in position 62: invalid continuation byte

../meson.build:1148:8: ERROR: Unhandled python exception

    This is a Meson bug and should be reported!

ERROR: meson setup failed

Because clang-15 is writing what seems to be binary data to stdout.

/usr/local/lib/carbon-collect.so:6:1061: warning: null character ignored [-Wnull-character]
<U+0000><U+0000><U+0000><U+0000><U+0000><84> <U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><84> <U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>Q<E5>td<U+0006><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0010><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>R<E5>td<U+0004><U+0000><U+0000><U+0000><U+0000><FF><U+000B><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><FF><U+000B><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><FF><U+000B><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>!<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>!<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0004><U+0000><U+0000><U+0000><U+0014><U+0000><U+0000><U+0000><U+0003><U+0000><U+0000><U+0000>GNU<U+0000><D9>m<AD><FE><U+001C><E0><86>[<EB>i<89>ݭ,<8E><A6><81><C6>q<F7><U+0000><U+0000><U+0000><U+0000><U+0007><U+0004><U+0000><U+0000><U+0003><U+0006><U+0000><U+0000>F<U+0001><U+0000><U+0000>\<U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><E6><U+0001><U+0000><U+0000><C0><U+0003><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>^<U+0003><U+0000><U+0000><FF><U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>*<U+0001><U+0000><U+0000><U+0019><U+0002><U+0000><U+0000>x<U+0004><U+0000><U+0000><U+000B><U+0003><U+0000><U+0000><A1><U+0005><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><CA><U+0003><U+0000><U+0000><88><U+0002><U+0000><U+0000><D4><U+0002><U+0000><U+0000><B9><U+0003><U+0000><U+0000><97><U+0004><U+0000><U+0000><U+0018><U+0002><U+0000><U+0000><DA><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><87><U+0003><U+0000><U+0000><83><U+0000><U+0000><U+0000><A7><U+0001><U+0000><U+0000>Q<U+0003><U+0000><U+0000><U+0001><U+0001><U+0000><U+0000><84><U+0005><U+0000><U+0000>#<U+0000><U+0000><U+0000><F3><U+0003><U+0000><U+0000>X<U+0003><U+0000><U+0000>T<U+0003><U+0000><U+0000><CA><U+0001><U+0000><U+0000><F5><U+0004><U+0000><U+0000>><U+0005><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><93><U+0000><U+0000><U+0000><C8><U+0004><U+0000><U+0000>><U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>p<U+0002><U+0000><U+0000>@<U+0002><U+0000><U+0000><C7><U+0001><U+0000><U+0000><D2><U+0002><U+0000><U+0000><B4><U+0003><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><82><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><C1><U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>0<U+0002><U+0000><U+0000>p<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>A<U+0004><U+0000><U+0000><U+001C><U+0001><U+0000><U+0000><FA><U+0001><U+0000><U+0000><U+0002><U+0005><U+0000><U+0000><F4><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>6<U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><BF><U+0004><U+0000><U+0000><U+0016><U+0004><U+0000><U+0000>z<U+0001><U+0000><U+0000><E4><U+0000><U+0000><U+0000>e<U+0005><U+0000><U+0000> <U+0003><U+0000><U+0000><E2><U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><B6><U+0004><U+0000><U+0000>H<U+0001><U+0000><U+0000>8<U+0004><U+0000><U+0000><EE><U+0005><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>}<U+0001><U+0000><U+0000><BF><U+0001><U+0000><U+0000><F6><U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>H<U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><93><U+0003><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0007><U+0000><U+0000><U+0000><F7><U+0004><U+0000><U+0000>C<U+0002><U+0000><U+0000><U+0014><U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><FC><U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>*<U+0003><U+0000><U+0000>8<U+0005><U+0000><U+0000><F4><U+0003><U+0000><U+0000><F9><U+0001><U+0000><U+0000>.<U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>I<U+0001><U+0000><U+0000><U+0016><U+0003><U+0000><U+0000>M<U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>L<U+0005><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>N<U+0005><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>q<U+0004><U+0000><U+0000><D0><U+0001><U+0000><U+0000><E9><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><91><U+0004><U+0000><U+0000>)<U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><88><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><95><U+0005><U+0000><U+0000><AE><U+0001><U+0000><U+0000>+<U+0003><U+0000><U+0000><98><U+0005><U+0000><U+0000>w<U+0004><U+0000><U+0000><U+0013><U+0003><U+0000><U+0000><F5><U+0005><U+0000><U+0000>)<U+0004><U+0000><U+0000><FF><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>d<U+0004><U+0000><U+0000><U+0003><U+0001><U+0000><U+0000><81><U+0005><U+0000><U+0000><AD><U+0004><U+0000><U+0000><D0><U+0005><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><BA><U+0000><U+0000><U+0000>D<U+0003><U+0000><U+0000><92><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><A3><U+0004><U+0000><U+0000>J<U+0000><U+0000><U+0000>-<U+0000><U+0000><U+0000><87><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>Y<U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>_<U+0004><U+0000><U+0000><B4><U+0000><U+0000><U+0000><C4><U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><FB><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><CA><U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><9B><U+0003><U+0000><U+0000>K<U+0002><U+0000><U+0000>7<U+0002><U+0000><U+0000>2<U+0001><U+0000><U+0000><D0><U+0004><U+0000><U+0000><C4><U+0003><U+0000><U+0000>e<U+0003><U+0000><U+0000>w<U+0002><U+0000><U+0000><U+001D><U+0005><U+0000><U+0000>|<U+0004><U+0000><U+0000><99><U+0004><U+0000><U+0000>u<U+0004><U+0000><U+0000>8<U+0000><U+0000><U+0000>.<U+0002><U+0000><U+0000>b<U+0003><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><C3><U+0003><U+0000><U+0000><ED><U+0005><U+0000><U+0000><E5><U+0002><U+0000><U+0000>X<U+0005><U+0000><U+0000><8B><U+0003><U+0000><U+0000>w<U+0001><U+0000><U+0000>W<U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>h<U+0002><U+0000><U+0000><D1><U+0002><U+0000><U+0000>9<U+0000><U+0000><U+0000><<U+0002><U+0000><U+0000><BD><U+0004><U+0000><U+0000><U+007F><U+0000><U+0000><U+0000>H<U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>Z<U+0005><U+0000><U+0000><U+000B><U+0004><U+0000><U+0000><FB><U+0002><U+0000><U+0000><E1><U+0002><U+0000><U+0000><U+0005><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>y<U+0005><U+0000><U+0000>H<U+0000><U+0000><U+0000><BB><U+0002><U+0000><U+0000>8<U+0002><U+0000><U+0000>[<U+0000><U+0000><U+0000>e<U+0002><U+00><U+0000><U+0000>r<U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><CF><U+0005><U+0000><U+0000><C6><U+0003><U+0000><U+0000><EE><U+0004><U+0000><U+0000><F4><U+0002><U+0000><U+0000><<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>a<U+0005><U+0000><U+0000>2<U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><80><U+0004><U+0000><U+0000>'<U+0003><U+0000><U+0000><AD><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><94><U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><E9><U+0004><U+0000><U+0000>|<U+0005><U+0000><U+0000>G<U+0000><U+0000><U+0000><8A><U+0001><U+0000><U+0000>5<U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><E3><U+0000><U+0000><U+0000><F2><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><8C><U+0003><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>)<U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><A9><U+0004><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000>k<U+0001><U+0000><U+0000><U+001B><U+0005><U+0000><U+0000><C2><U+0000><U+0000><U+0000>5<U+0002><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><CE><U+0001><U+0000><U+0000><U+0000><U+0000><U+0000><U+0000><U+007F><U+0005><U+0000><U+0000><ED><U+0002><U+0000><U+0000><A8><U+0003>

I had to use strace -v -f --trace=execve,execveat -s 100000000 ./configure to figure out what was going on (the command being run by meson was approximately clang-15 "-m64" "-mcx16" ~/Backups/testfile.c "-E" "-P" "-Xclang" "-Wl,--start-group" "-load" "-Xclang" "/usr/local/lib/carbon-collect.so" "-Wl,--end-group" "-Xclang" "-add-plugin" "-Xclang" "carbon-collect" "-Xclang" "-plugin-arg-carbon-collect" "-Xclang" "/home/aeden/qemu-upstream" "-Xclang" "-plugin-arg-carbon-collect" "-Xclang" "/home/aeden/qemu-upstream" "-D_FILE_OFFSET_BITS=64" "-P" "-O0" "-Werror=implicit-function-declaration" "-std=gnu11")

By the way, I noticed the following comment in QEMU/meson.build:

# Specify linker-script with add_project_link_arguments so that it is not placed
# within a linker --start-group/--end-group pair

Unfortunately I can't use add_project_link_arguments in this case, because the extra command-line arguments need to be passed to clang when the object file is built, not at the final linking step...

My workaround was to delete the following lines in mesonbuild/compilers/mixins/clike.py:

            if group_start >= 0:
                # Last occurrence of a library
                new.insert(group_end + 1, '-Wl,--end-group')
                new.insert(group_start, '-Wl,--start-group')
aleden commented 4 months ago

Here are the changes I've settled on:

diff --git a/mesonbuild/compilers/mixins/clike.py b/mesonbuild/compilers/mixins/clike.py
index 76c8e0413..c5e727579 100644
--- a/mesonbuild/compilers/mixins/clike.py
+++ b/mesonbuild/compilers/mixins/clike.py
@@ -70,21 +70,29 @@ class CLikeCompilerArgs(arglist.CompilerArgs):
         if copy:
             new = self.copy()
         else:
             new = self
         # This covers all ld.bfd, ld.gold, ld.gold, and xild on Linux, which
         # all act like (or are) gnu ld
         # TODO: this could probably be added to the DynamicLinker instead
         if isinstance(self.compiler.linker, (GnuLikeDynamicLinkerMixin, SolarisDynamicLinker, CompCertDynamicLinker)):
             group_start = -1
             group_end = -1
+            ignore = False
             for i, each in enumerate(new):
+                if ignore:
+                    ignore = False
+                    continue
+                if each == "-Xclang":
+                    ignore = True
+                    continue
+
                 if not GROUP_FLAGS.search(each):
                     continue
                 group_end = i
                 if group_start < 0:
                     # First occurrence of a library
                     group_start = i
             if group_start >= 0:
                 # Last occurrence of a library
                 new.insert(group_end + 1, '-Wl,--end-group')
                 new.insert(group_start, '-Wl,--start-group')
wallabra commented 2 weeks ago

@aleden this almost looks like PR material, although I suppose in that case it would have to be configurable instead of hard-coding "-Xclang". But if we go down the configurability rabbit hole, we might want spans larger than "the option immediately after this"...

aleden commented 2 weeks ago

@wallabra What would you suggest? Submit this as a PR anyway to get a discussion going?

wallabra commented 2 weeks ago

Sounds good to me.