search

mesonbuild / meson

The Meson Build System
http://mesonbuild.com
Apache License 2.0
5.6k stars 1.63k forks source link

macOS frameworks are not found by mesonbuild: `extraframework` method fails #13608

Open barracuda156 opened 2 months ago

barracuda156 commented 2 months ago

Describe the bug Looks like the bug was never fixed since 2019: https://github.com/mesonbuild/meson/issues/5070 The current version of meson still cannot find macOS frameworks via extraframework method:

Run-time dependency applewm found: YES 1.4.1
Run-time dependency xfixes found: YES 6.0.1
Run-time dependency x11 found: YES 1.8.10
hw/xquartz/pbproxy/meson.build:13: WARNING: Project targets '>= 0.47.0' but uses feature introduced in '0.50.0': include_directories kwarg of type string. Use include_directories('..') instead
Run-time dependency cocoa found: NO (tried framework)

hw/xquartz/pbproxy/meson.build:23:8: ERROR: Dependency "Cocoa" not found, tried framework

To Reproduce Try configure xorg-server with meson and gcc.

Expected behavior There is no reason why this should fail with gcc. Compiler itself of course supports linking to macOS frameworks.

system parameters

christophecvr commented 2 months ago

Hello I indeed did had same issues when building gstreamer1-plugins-bad package. Did found the cause of error and I'm just to the point I wanted to create a first pull request with solution. However I did not used gcc but the apple's xcode \

The error occurs when searching for FrameWorks who do contain c,cxx,cpp, ObjC and ObjCxx combined. The FrameWork here in the error case is CoreFoundation is present in the CommandlineTools and in The xcode full version tools.

Messon-log extract from error : 

Finding framework path by running:  /usr/bin/clang++ -v -E - -pipe -Os -std=gnu++17 -stdlib=libc++
-isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -I/opt/local/include
-L/opt/local/lib -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk

CMake binary for host machine is cached.
Preliminary CMake check failed. Aborting.
Run-time dependency corefoundation found: NO (tried pkgconfig, framework and cmake)

sys/applemedia/meson.build:42:21: ERROR: Dependency "CoreFoundation" not found, tried pkgconfig, framework and cmake

Further analyze of command did revealed that the used command failed. \ command issued by meson to find library used in terminal :

/usr/bin/clang++ -v -E - -pipe -Os -std=gnu++17 -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -I/opt/local/include -L/opt/local/lib -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk

Issued an error and the command did not returned 0 cause it did not find any frameWork.

Apple clang version 12.0.0 (clang-1200.0.32.29)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
clang: warning: argument unused during compilation: '-L/opt/local/lib' [-Wunused-command-line-argument]
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -Werror=implicit-function-declaration -E -disable-free -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mthread-model posix -mframe-pointer=all -fno-strict-return -masm-verbose -munwind-tables -target-sdk-version=10.15.6 -fcompatibility-qualified-id-block-type-checking -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 609.8 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I /opt/local/include -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/local/include -internal-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include -Os -Wno-reorder-init-list -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-final-dtor-non-final-class -Wno-extra-semi-stmt -Wno-misleading-indentation -Wno-quoted-include-in-framework-header -Wno-implicit-fallthrough -Wno-enum-enum-conversion -Wno-enum-float-conversion -std=gnu++17 -fdebug-compilation-dir /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks -ferror-limit 19 -fmessage-length 231 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fobjc-runtime=macosx-10.15.0 -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o - -x c -
error: invalid argument '-std=gnu++17' not allowed with 'C'

I did found the cause of this issue and thought it was only clang related but guess now it is macos related with any compiler.

My solution to this problem is to adapt the specific FrameWork Find command into file : 

        # TODO: this really needs to be *AppleClang*, not just any clang.
        if self.id != 'clang':
            raise mesonlib.MesonException('Cannot find framework path with non-clang compiler')
        # Construct the compiler command-line
        commands = self.get_exelist(ccache=False) + ['-v', '-E', '-']
        commands += self.get_always_args()
        # Add CFLAGS/CXXFLAGS/OBJCFLAGS/OBJCXXFLAGS from the env
        commands += env.coredata.get_external_args(self.for_machine, self.language)
        mlog.debug('Finding framework path by running: ', ' '.join(commands), '\n')
        os_env = os.environ.copy()
        os_env['LC_ALL'] = 'C'
        _, _, stde = mesonlib.Popen_safe(commands, env=os_env, stdin=subprocess.PIPE)
        paths: T.List[str] = []
        for line in stde.split('\n'):
            if '(framework directory)' not in line:
                continue
            # line is of the form:
            # ` /path/to/framework (framework directory)`
            paths.append(line[:-21].strip())
        return paths

Into : 

       # TODO: this really needs to be *AppleClang*, not just any clang.
        if self.id != 'clang':
            raise mesonlib.MesonException('Cannot find framework path with non-clang compiler')
        # Construct the compiler command-line
        commands = self.get_exelist(ccache=False) + ['-v', '-E', '-']
        commands += self.get_always_args()
        # Add CFLAGS/CXXFLAGS/OBJCFLAGS/OBJCXXFLAGS from the env
        invalidCargs = ['-std=c++11', '-std=gnu++11', '-std=c++17', '-std=gnu++17']
        # Do not add invalid arguments in 'C' for this specific command. They do break this command with msg:
        # error: invalid argument '-std=<arg>' not allowed with 'C'
        # The result is not find FrameWork Library while it is present on system or -isysroot links.
        commands += [x for x in env.coredata.get_external_args(self.for_machine, self.language) if x not in invalidCargs]
        mlog.debug('Finding framework path by running: ', ' '.join(commands), '\n', 'Without the not allowed arguments with C : ', ' '.join(invalidCargs), '\n')
        os_env = os.environ.copy()
        os_env['LC_ALL'] = 'C'
        _, _, stde = mesonlib.Popen_safe(commands, env=os_env, stdin=subprocess.PIPE)
        paths: T.List[str] = []
        for line in stde.split('\n'):
            if '(framework directory)' not in line:
                continue
            # line is of the form:
            # ` /path/to/framework (framework directory)`
            paths.append(line[:-21].strip())
        return paths

@barracuda156

I had the same issue but when using clang. So do not know if my solution above does solve you're similar issue. I was just to the point to submit a patch to meson and a temporary one for macports. But since this issue by me occurs in the clang section of clike.py do not know if my solution works when using gcc can you let it me know ?

Thanks

In the mean time I wait on the response before I continue to create a correct pull request.

barracuda156 commented 2 months ago 
invalidCargs = ['-std=c++11', '-std=gnu++11', '-std=c++17', '-std=gnu++17']

Perhaps it should include all C++ std-type args? There are also 03, 14, 20 and 23. (And then can it be done in a generic form, so that we do not need to keep adding upcoming standards?)

christophecvr commented 2 months ago

@barracuda156 Indeed I thought about that also , but some of the -std= are accepted in the ObjC and ObjCxx so I just do not know for 100 % sure that they are not required to find some particular headers embedded into the FrameWork. That is why I (to start at least) just created a invalidCargs list which can be expanded. But for sure if we just can exclude all flags starting with -std= this would solve it overall at once.

It would well be nice if You try by you build with gcc if my patch does solve you're problem (guess you well have to extend the invalidCargs list ) So I know that it will apply to any compiler and not only to clang.

In the mean time I look to modify an try out a similar solution by excluding all -std= by me.

My personal tempo modified macports meson modified port with patch can be found at :

https://github.com/christophecvr/macports-ports/tree/cvrquartzx11/python/py-meson

the extra meson patch file is :

https://github.com/christophecvr/macports-ports/blob/cvrquartzx11/python/py-meson/files/patch-meson-unbreak-framework-find.diff

christophecvr commented 2 months ago

@barracuda156 oeps corrected the web-links in previous comment.

barracuda156 commented 2 months ago

@christophecvr Thank you and sorry for a delay. Unfortunately, it does not work for me.

Looks like meson simply does not even try to look into any sane locations for frameworks:

Run-time dependency x11 found: YES 1.8.10
Finding framework path by running:  /opt/local/bin/gcc-mp-14 -v -E - -pipe -Os -fobjc-exceptions -Wno-error=return-type -arch ppc -isystem/opt/local/include/LegacySupport -I/opt/local/include -I/opt/local/libexec/dispatch/usr/include 
 Without the not allowed arguments with C :  -std=c++11 -std=gnu++11 -std=c++17 -std=gnu++17 

Run-time dependency cocoa found: NO (tried framework)

hw/xquartz/pbproxy/meson.build:21:8: ERROR: Dependency "Cocoa" not found, tried framework

In fact a mere -framework Cocoa would have worked.

christophecvr commented 2 months ago

Hi no problem for the delay, I self stopped working yesterday and only restarted this morning. And .. Just had the unpleasant surprise that the update of 6 hours ago of py-setuptools does break the g-ir-scanner . So what all builds of packages who requires Gobjects and require the use of g-ir-scanner do fail now with msg

n <module> from distutils.msvccompiler import MSVCCompiler ModuleNotFoundError: No module named 'distutils.msvccompiler'

In you're log I can see that meson also use same framework find command with other compiler then clang. By me Cocoa framework is in path's : 

/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks
/System/Library/Frameworks

When I solved the issue with upgraded py-setuptools I also will install gcc14 and check here what it does. But I already checked the port xorg-server and it does not have a external CXXFLAG who breaks the meson framework find command. This means that my patch is not needed for that package. but did you not tried to add to macports and extra cflag -framework Cocoa

with in portfile an extra line :

configure.cflags-append -framework Cocoa

barracuda156 commented 2 months ago

But I already checked the port xorg-server and it does not have a external CXXFLAG who breaks the meson framework find command. This means that my patch is not needed for that package. but did you not tried to add to macports and extra cflag -framework Cocoa

@christophecvr Ok, a few notes on this to keep it clear. MacPorts does not build xorg-server with meson at all, and the version which I am building is different from what MacPorts has. So portfiles in the master are irrelevant. xorg-server uses extraframework stuff in two files, I think, like this: https://github.com/XQuartz/xorg-server/blob/0ea9b595891f2f31915538192961f3404d9ca699/hw/xquartz/mach-startup/meson.build#L29-L39 This is what fails to work correctly.

I just used xorg-server as an example of the issue, the problem which I hope to get solved is with meson. For xorg-server you could add linker flags to portfile, or you could patch meson.build files (which is needed anyway to get rid of defunct methods) to use link_args to pass needed flags (which is what I did, but your suggestion will also work, of course, just flags will be passed on every instance of linking something during the build.

christophecvr commented 2 months ago

@barracuda156 Oh yes did not check which build system macports port xorg-server uses. But yes there are many differences by you cause you're have macos 10.6 think @rpath is also not working .

But On you're link above I noticed that they set : 

x11_bin_deps = [
     meson.get_compiler('c').find_library('Xplugin'),
     dependency('Carbon', method: 'extraframework'),
     cocoa,
     dependency('CoreAudio', method: 'extraframework'),
     dependency('IOKit', method: 'extraframework')
]

Must this not be : 

x11_bin_deps = [
     meson.get_compiler('c').find_library('Xplugin'),
     dependency('Carbon', method: 'extraframework'),
     dependency('Cocoa', method: 'extraframework'),
     dependency('CoreAudio', method: 'extraframework'),
     dependency('IOKit', method: 'extraframework')
]

?

barracuda156 commented 2 months ago

Apparently they define it here: https://github.com/XQuartz/xorg-server/blob/0ea9b595891f2f31915538192961f3404d9ca699/hw/xquartz/pbproxy/meson.build#L23 Don’t ask me why :)

christophecvr commented 2 months ago

Apparently they define it here: https://github.com/XQuartz/xorg-server/blob/0ea9b595891f2f31915538192961f3404d9ca699/hw/xquartz/pbproxy/meson.build#L23 Don’t ask me why :)

Well yes, but I guess that at configure time this maybe not works. I would give it a try by patching that line to : dependency ... maybe it works then.

christophecvr commented 2 months ago

@barracuda156

But in error log on top I just did see that the error occurs while configuring :

/hw/xquartz/pbproxy

and there they used : cocoa = dependency('Cocoa', method: 'extraframework')

so ??

barracuda156 commented 2 months ago

@christophecvr Exactly, so the method itself does not work.

barracuda156 commented 2 months ago

@christophecvr By the way, GIR scanner still broken…

https://github.com/macports/macports-ports/pull/25464#issuecomment-2323373290

christophecvr commented 2 months ago

@christophecvr By the way, GIR scanner still broken…

macports/macports-ports#25464 (comment)

@barracuda156 Hello , now that we are rid of the broken scanner issue i took the time to review all the patches implemented by meson in the python/py-meson port

I discovered a not coherent issue.

There was already a patch implemented to remove a blockage into the original : mesonbuild/compilers/mixins/clike.py file.

The port patch file who does that is :

python/py-meson/files/patch-meson-gcc-appleframeworks.diff

It does remove the clang limitation to use that function to obtain the right framework implemented by meson self.

Since you're using gcc according to meson that function may not be used. maybe this should be reset like meson did and then hope that perhaps the other way using extraframeworks method will then work ???.

I wel would anayway change the line 

x11_bin_deps = [
     meson.get_compiler('c').find_library('Xplugin'),
     dependency('Carbon', method: 'extraframework'),
     cocoa,
     dependency('CoreAudio', method: 'extraframework'),
     dependency('IOKit', method: 'extraframework')
]

to 

x11_bin_deps = [
     meson.get_compiler('c').find_library('Xplugin'),
     dependency('Carbon', method: 'extraframework'),
     dependency('Cocoa', method: 'extraframework'),
     dependency('CoreAudio', method: 'extraframework'),
     dependency('IOKit', method: 'extraframework')
]

My patch to the command for only clang (according to meson) is I think only required for clang self and is an apples clang issue.

Try once to build meson without the patch : python/py-meson/files/patch-meson-gcc-appleframeworks.diff

but well with the modification concerning cocoa in you're project xorg-server

There is a chance it will work I hope.

christophecvr commented 2 months ago

@barracuda156 I did further research about this issue.

Knowing this I adapted a bit in meson (py-meson macports) . I got it so far that it does find the framework path with gcc but then when going further we start with framework.py module and there it stops reverts back to cmake method and bugs( i guess here it is a macports issue who brakes cmake)

Anyway for to day i've had enough.

If you're interested I pushed to my clone of macports in branch cvrquartzx11 a temporary save for meson py-meson build on macports.

patch patch-meson-gcc-appleframeworks.diff is commented and replaced (temporary) by cvr-meson-framework-find-compilers.patch

see :

https://github.com/christophecvr/macports-ports/tree/cvrquartzx11/python/py-meson

It does has extra debug lines starting with CVR. Yes that is over debug but needed to trace the whole configure process for now. I added the CVR in front so it will be easy to clean up later on