search

mesonbuild / meson

The Meson Build System
http://mesonbuild.com
Apache License 2.0
5.64k stars 1.64k forks source link

Meson passing extra linker options to ninja backend (-Wl,--allow-shlib-undefined) #7506

Open beyondsociety opened 4 years ago

beyondsociety commented 4 years ago

I am using meson to compile my own operating system that I have been working on and while tracking down a linking issue with my code, I started looking at what meson and ninja was doing in the background to find the problem. I noticed by default ninja passes the linker options -Wl,--as-needed -Wl,--no-undefined which can be turned off with the base options b_asneeded and b_lundef. If I set both of these options to false, they get removed from the command line that ninja invokes for the linker through gcc but then passes -Wl,--allow-shlib-undefined instead which can't be turned off. Code compiles fine with this linker option but since I am creating my own os, I want complete control over my build options so I'd like for meson to not add any extra options.

Describe the bug Meson passes extra linker option -Wl,--allow-shlib-undefined to ninja backend when it shouldn't. I am using a cross-compiler that is built for osdev (created from the osdev wiki: https://wiki.osdev.org/GCC_Cross-Compiler) and the only library that gets built for the cross-compiler is libgcc which is a static library. Looking up this linker option, it says that it is a default option for shared libraries which makes no sense since I am only compiling the one static library with my code.

To reproduce Meson-log Cross-compiler.build Compile-output

Link to my repo: Ibox

Expected behavior I would expect this option to not be enabled by default, but it is for some reason. Ideal behavior would be for it to not get linked in automatically and instead be turned on based on buildtype supplied to meson. The same exact behavior when I set buildtype=plain which doesn't pass any gcc extra options would be ideal.

Extra info Trying to track down the issue, I found the linker option is getting set in compilers.py under the code to deal with apples bitcode:

    # Apple's ld (the only one that supports bitcode) does not like any  
    # -undefined arguments at all, so don't pass these when using bitcode
    if not bitcode:
        if (not is_shared_module and
                option_enabled(linker.base_options, options, 'b_lundef')):
            args.extend(linker.no_undefined_link_args())
        else:
            args.extend(linker.get_allow_undefined_link_args()) # This line calls the function in linkers.py

If I comment out this line, it removes it from the ninja backend. I took code for the base options and created another b_option to turn this off optionally and was successful in doing so. Not sure what the best option for this is or whether adding one more option would be ok as there are already a lot of options for meson as it is. I would love to know why this gets called and how to remove it.

System parameters

dcbaker commented 4 years ago

this is a feature, not a bug. Meson always explicitly adds one option or the other (as they do exactly opposite things) so that we're not relying on the default from your compiler, which at least for gcc can be configured to default to either at compile time.

beyondsociety commented 4 years ago

I thought so, would we be able to add another base option to meson that toggles this on or off? I would be able to help with this, I already have a proof of concept to see if this would work or not.

On Thu, Jul 30, 2020, 12:24 PM Dylan Baker notifications@github.com wrote:

this is a feature, not a bug. Meson always explicitly adds one option or the other (as they do exactly opposite things) so that we're not relying on the default from your compiler, which at least for gcc can be configured to default to either at compile time.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mesonbuild/meson/issues/7506#issuecomment-666581845, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACU7RGQ5FHM2U2AZXYQW6MLR6G3HVANCNFSM4PKA3B2Q .

dcbaker commented 4 years ago

I'm just not sure why you would explicitly want to fall back to letting the compiler/linker default to some unknown value. However, there has been talk about adding a 'default' value for other cases like this, and we do allow this already for PIC through a historical accident. @jpakkane, opinions?

beyondsociety commented 4 years ago

This makes sense for a hosted/userspace environment but my situation is different as I am compiling in a freestanding environment where there is no c library or extra libraries and options compiled in for osdev. I would like to have full control over my build options like if I was building for baremetal. I am using a cross-compiled gcc and clang to build my os.

I was thinking it would be nice to have the same functionality as buildtype=plain where nothing gets added to the gcc compiler but also for the linker and only make this a default option for the linker if buildtype=release.

On Thu, Jul 30, 2020, 1:31 PM Dylan Baker notifications@github.com wrote:

I'm just not sure why you would explicitly want to fall back to letting the compiler/linker default to some unknown value. However, there has been talk about adding a 'default' value for other cases like this, and we do allow this already for PIC through a historical accident. @jpakkane https://github.com/jpakkane, opinions?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mesonbuild/meson/issues/7506#issuecomment-666634569, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACU7RGUUSDC67RQIOQC2XRTR6HDBFANCNFSM4PKA3B2Q .

siradam7th commented 4 years ago

I just ran into the same problem, i'm trying to link a UEFI application which is freestanding and it is failing to link because meson generates a ninja build file that passes an extra "-Wl,/release" to the linker (i'm using lld-link, llvm's linker), so it is causing it to fail with a lld-link: error: could not open '/release': no such file or directory Since meson supports cross-compiling, this is a must have feature, the ability to instruct meson to not add any additional linker arguments other than the ones specified in 'c_link_args'. I also suggest taking what @beyondsociety suggested even further and make such change affect all the buildtypes.

beyondsociety commented 4 years ago

I have been wanting to submit a pull request to fix this issue but haven't heard back yet on which method is the best choice for meson. I am also using clang/llvm to test with my os and using llvm-lld to link but don't have your release error or have tried to compile my os with llvm-link yet. Are you passing buildtype=release to meson in your default-options or just using the default settings meson passes? Would be helpful if we could see your meson.build file or at least the meson-log to see where the issue lies.

On Sat, Aug 22, 2020 at 8:01 PM Billel Adam notifications@github.com wrote:

I just ran into the same problem, i'm trying to link a UEFI application which is freestanding and it is failing to link because meson generates a ninja build file that passes an extra "-Wl,/release" to the linker (i'm using lld-link, llvm's linker), so it is causing it to fail with a lld-link: error: could not open '/release': no such file or directory Since meson supports cross-compiling, this is a must have feature, the ability to instruct meson to not add any additional linker arguments other than the ones specified in 'c_link_args'. I also suggest taking what @beyondsociety https://github.com/beyondsociety suggested even further and make such change affect all the buildtypes.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mesonbuild/meson/issues/7506#issuecomment-678719110, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACU7RGV2Z56PWXDU64JYG2DSCB2A7ANCNFSM4PKA3B2Q .

TheMadHau5 commented 3 years ago

This bug heavily affects lld on MinGW, even causing build failures (lld: error: unknown argument: --allow-shlib-undefined) as that option is not supported by ld.lld compiled for Windows. This requires me to manually edit build.ninja each time meson.build is updated, which is tedious and why I consider this behaviour to be a bug. A solution would be greatly appreciated, thanks.

dcbaker commented 3 years ago

The correct thing to do is a compile test the first time we try to get --allow-shlib-undefined. The problem is that the output format determines whether the flag is valid or not. For mach-o and elf (mac and unix) it is, for pe and coff (windows) and web-assembly it isn't.

TheMadHau5 commented 3 years ago

Yeah. I'm aware of that and I was thinking that a check could be added such that if Meson is being run under Windows (natively or via any of MinGW, MSYS, or Cygwin) and if so, that argument would simply not be output. Maybe something like this:

        # mesonbuild/linkers.py, line 777
        self.has_allow_shlib_undefined = not (mesonlib.is_windows() or mesonlib.is_cygwin() or 'unknown argument: --allow-shlib-undefined' in e)

as the existing check doesn't seem to be working correctly because self.exelist is ['clang'], which doesn't give the expected error (however, running ld.lld --allow-shlib-undefined gives the proper error, as expected).

TheMadHau5 commented 3 years ago

On quick testing, my edits seem to have worked, but I'm not sure if they use good programming practices for Python and are up to the standards of Meson. Thus, I'll release the changes under public domain, so someone can submit a pull request based on them. (Or better yet, a pull request that adds a b_shlib_undefined flag, set to true by default, which can be turned off in the build file)

dcbaker commented 3 years ago

We really just need to move the has_argument test code into the compiler so we can use it there... hmmm

1480c1 commented 3 years ago

Any updates on this issue? I recently encountered it when trying to compile fontconfig with clang and ld.lld with mingw-w64 from msys2 and attempting to enable b_lundef or b_asneeded does nothing so I don't really have a way to disable it without modifying meson itself