search

microsoft / vcpkg

C++ Library Manager for Windows, Linux, and MacOS
MIT License
22.93k stars 6.33k forks source link

Enhancement: Invoke meson with additional include and library paths for BSD systems #39748

Closed ChihHao-Su closed 2 months ago

ChihHao-Su commented 3 months ago

Locations for putting user libraries and include files used by BSD systems:

Meson doesn't find libraries and includes inside these directories (e.g. for OpenBSD, user libraries and includes are inside /usr/local/lib and /usr/local/include), and this causes you cannot get an out-of-box installation for many package that uses meson to build, for example, glib. These are logs of vcpkg install glib, on OpenBSD: Console output:

-- Configuring x64-openbsd-dbg
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:127 (message):
    Command failed: /usr/local/bin/python3 -I /home/niko/vcpkg/downloads/tools/meson-1.3.2-d646d3/meson.py -Dselinux=disabled -Dlibmount=disabled -Dgtk_doc=false -Dinstalled_tests=false -Dlibelf=disabled -Dman=false -Dtests=false -Dxattr=false --backend ninja --wrap-mode nodownload -Dbuildtype=plain -Doptimization=plain --libdir lib --native /home/niko/vcpkg/buildtrees/glib/meson-x64-openbsd-dbg.log -Ddebug=true --prefix /home/niko/vcpkg/packages/glib_x64-openbsd/debug --includedir ../include -Dcmake_prefix_path=['/home/niko/vcpkg/installed/x64-openbsd/debug','/home/niko/vcpkg/installed/x64-openbsd'] /home/niko/vcpkg/buildtrees/glib/src/glib-2-322e03f702.clean
    Working Directory: /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg
    Error code: 1
    See logs for more information:
      /home/niko/vcpkg/buildtrees/glib/config-x64-openbsd-dbg-meson-log.txt.log
      /home/niko/vcpkg/buildtrees/glib/config-x64-openbsd-dbg-out.log

Call Stack (most recent call first):
  installed/x64-openbsd/share/vcpkg-tool-meson/vcpkg_configure_meson.cmake:436 (vcpkg_execute_required_process)
  ports/glib/portfile.cmake:42 (vcpkg_configure_meson)
  scripts/ports.cmake:192 (include)

error: building glib:x64-openbsd failed with: BUILD_FAILED
Elapsed time to handle glib:x64-openbsd: 1.1 min
Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
  https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+glib
You can submit a new issue at:
  https://github.com/microsoft/vcpkg/issues/new?title=[glib]+Build+error+on+x64-openbsd&body=Copy+issue+body+from+%2Fhome%2Fniko%2Fvcpkg%2Finstalled%2Fvcpkg%2Fissue_body.md

Part of config-x64-openbsd-dbg-out.log:

Checking if "stack grows check" runs: NO (1)
Running compile:
Working directory:  /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmp8pv3xc97
Code:
 #include <iconv.h>

int main() {
    iconv_open("","");
}
-----------
Command line: `/usr/bin/clang++ -fPIC -g -L/home/niko/vcpkg/installed/x64-openbsd/debug/lib -I/home/niko/vcpkg/installed/x64-openbsd/include /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmp8pv3xc97/testfile.cpp -o /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmp8pv3xc97/output.exe -fPIC -g -D_FILE_OFFSET_BITS=64 -O0 -fpermissive -Werror=implicit-function-declaration -fuse-ld=/usr/bin/ld.lld` -> 1
stderr:
/home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmp8pv3xc97/testfile.cpp:1:10: fatal error: 'iconv.h' file not found
#include <iconv.h>
         ^~~~~~~~~
1 error generated.
-----------
Running compile:
Working directory:  /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmpxju7wrxo
Code:

        #ifdef __has_include
         #if !__has_include("iconv.h")
          #error "Header 'iconv.h' could not be found"
         #endif
        #else
         #include <iconv.h>
        #endif
-----------
Command line: `/usr/bin/clang++ -fPIC -g -I/home/niko/vcpkg/installed/x64-openbsd/include /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmpxju7wrxo/testfile.cpp -E -P -fPIC -g -D_FILE_OFFSET_BITS=64 -P -O0 -fpermissive -Werror=implicit-function-declaration` -> 1
stderr:
/home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmpxju7wrxo/testfile.cpp:4:12: error: "Header 'iconv.h' could not be found"
          #error "Header 'iconv.h' could not be found"
           ^
1 error generated.
-----------
Running compile:
Working directory:  /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmpm8y7hhad
Code:
 int main(void) { return 0; }

-----------
Command line: `/usr/bin/clang++ -fPIC -g -L/home/niko/vcpkg/installed/x64-openbsd/debug/lib -I/home/niko/vcpkg/installed/x64-openbsd/include /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmpm8y7hhad/testfile.cpp -o /home/niko/vcpkg/buildtrees/glib/x64-openbsd-dbg/meson-private/tmpm8y7hhad/output.exe -fPIC -g -D_FILE_OFFSET_BITS=64 -O0 -fpermissive -Werror=implicit-function-declaration -Wl,--start-group -liconv -Wl,--end-group -fuse-ld=/usr/bin/ld.lld -Wl,--allow-shlib-undefined -fuse-ld=/usr/bin/ld.lld` -> 1
stderr:
ld.lld: error: unable to find library -liconv
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

...

Run-time dependency iconv found: NO (tried builtin and system)

../src/glib-2-322e03f702.clean/meson.build:2082:13: ERROR: Dependency "iconv" not found, tried builtin and system

There's an issues mention mentioned this problem: https://github.com/mesonbuild/meson/issues/4468, but it seems that they are still debating. I believe it might be better if we pass these BSD locations to meson (set additional include path and library path), in order to provide an out-of-box installation of the packages use meson, as a temporary solution?

dg0yt commented 3 months ago

There are two perspectives on your problem:

  1. Dealing with /usr/local
  2. Dealing with iconv and gettext.

The first point could probably be adressed with a machine file, and this is generated in vcpkg anyways. Maybe you can already configure this in your bsd triplet files. (I'm not convinced that /usr/local should generally be added.)

The second point is whether meson does enough to find iconv and intl. TBH I don't know if vcpkg should build them or find them in the system for BSD. However, I have a PR here which chose iconv and intl as end-to-end tests for dependency detection in meson, and the result wasn't encouraging: https://github.com/microsoft/vcpkg/pull/38658 You are encouraged to check what it does for BSD.

ChihHao-Su commented 3 months ago

@dg0yt Sorry, I may not have made it clear. Here I just described one problem -- Meson won't find header files and library files under BSD locations (/usr/local is what OpenBSD uses). This problem causes packages use meson to build, cannot find anything installed with system package manager (the example I gave is glib cannot find iconv). If I execute

export LIBRARY_PATH=/usr/local/lib
export CPATH=/usr/local/include

And then run vcpkg install glib, everything work fine.

In this issue, what I'm doing is to make a discussion about should we solve the problem on the layer of vcpkg .

SchaichAlonso commented 2 months ago

@dg0yt Sorry, I may not have made it clear. Here I just described one problem -- Meson won't find header files and library files under BSD locations (/usr/local is what OpenBSD uses). This problem causes packages use meson to build, cannot find anything installed with system package manager (the example I gave is glib cannot find iconv). If I execute

export LIBRARY_PATH=/usr/local/lib
export CPATH=/usr/local/include

And then run vcpkg install glib, everything work fine.

In this issue, what I'm doing is to make a discussion about should we solve the problem on the layer of vcpkg .

@dg0yt is right, iconv is it's own issue. OpenBSD has BSD's implementation of iconv, but glib and friends - via gnu gettext - rely on GNU's implementation of iconv. Vcpkg actually has a port for GNU's iconv, just that it omits deployment on OpenBSD.

The line omitting the deployment of GNU iconv, rather then whitelisting Windows, MacOS and Android, probably should blacklist glibc based linux implementations which have iconv in the libc. But even if they have it in the libc, we should be able to just ship and use our own one even there.

SchaichAlonso commented 2 months ago

Reproducting Github Action using ChristopherHX/github-act-runner on a vultr hosted OpenBSD-7.5: link Notoriously, the artifact for ${VCPKG_ROOT}/installed/vcpkg/info indicates that the libiconv package did not provision the iconv.h file that causes glib consumes.

dg0yt commented 2 months ago

OpenBSD has BSD's implementation of iconv, but glib and friends - via gnu gettext - rely on GNU's implementation of iconv.

Where can this be verified w.r.t. BSD?

FTR we also to deal musl libc Linux systems which don't have gettext/libintl in the C lib.

I do know that gettext comes with detection logic built into m4 files for autotools, and I assume that's why we don't have pkg-config files. CMake has Find modules, but they need wrappers in vcpkg. Meson ... still not encouraging.

ChihHao-Su commented 2 months ago

Thank you, I see. Let's divide this issue into two.