Incorrect PKG_CONFIG_PATH path separator with meson

star-hengxing commented 1 month ago

Xmake Version

dev

Operating System Version and Architecture

Windows

Describe Bug

在 Windows 上构建 meson 包，如果需要用 pkgconfig 传递依赖信息，meson 有可能会找到 msys2 上的 pkgconfig 而不是 xrepo 的。 xmake windows 包的 .pc 使用 ; 作为分隔符，此时因为 https://github.com/mesonbuild/meson/issues/9714 可能会解析 PKG_CONFIG_PATH 失败，最终导致找不到依赖库。

比如依赖了 glib 的 meson 包：

Called: `C:\Users\star\scoop\apps\msys2\2024-07-27\usr\bin\pkgconf.EXE --modversion glib-2.0` -> 1
stderr:
Package glib-2.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `glib-2.0.pc'
to the PKG_CONFIG_PATH environment variable
Package 'glib-2.0', required by 'virtual:world', not found

Called: `C:\Users\star\AppData\Local\.xmake\packages\p\pkgconf\2.0.3\be5c11cb1b754512913eca4b14a1fb60\bin\pkgconf.EXE --modversion glib-2.0` -> 0
stdout:
2.78.1

暂时找到的 workaround 是把 msys2 的 pkgconfig rename 让 meson 找不到。

Expected Behavior

N/A

Project Configuration

N/A

Additional Information and Error Logs

N/A

waruqi commented 1 month ago

是指这里么? https://github.com/xmake-io/xmake/blob/d342e78c2f24434935eb371d0dd5bba5410dbf3f/xmake/modules/package/tools/meson.lua#L485

必须得用 :?

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Do you mean here? https://github.com/xmake-io/xmake/blob/d342e78c2f24434935eb371d0dd5bba5410dbf3f/xmake/modules/package/tools/meson.lua#L485

Must use :?

star-hengxing commented 1 month ago

修改 xmake 源码，做了几个测试：

xrepo 的 pkgconfig 必须要用 ; 分隔，用 : 会报错。
msys2 的 pkgconfig 必须要用 : 分隔，用 ; 会报错。

感觉好像 meson 和 xmake 的行为都没啥问题，但我的 PATH 里有 C:\Users\star\scoop\apps\msys2\2024-07-27\mingw64\bin，所以 meson 优先去找了 msys2 pkgconf。怎么让他不用 msys2 的呢，比如清理一下 PATH 里的 msys2 相关路径？

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Modify the xmake source code and do several tests:

The pkgconfig of xrepo must be separated by ;, and an error will be reported if : is used.
The pkgconfig of msys2 must be separated by :, and an error will be reported if ; is used.

It seems that there is no problem with the behavior of meson and xmake, but there is C:\Users\star\scoop\apps\msys2\2024-07-27\mingw64\bin in my PATH, so meson looks for msys2 first. pkgconf. How to prevent him from using msys2, for example, clearing the msys2 related paths in PATH?

star-hengxing commented 1 month ago

刚刚发现，一些用 pkgconf 找依赖的 cmake 包也会有这个问题。

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

I just discovered that some cmake packages that use pkgconf to find dependencies will also have this problem.

waruqi commented 1 month ago

msys的bin 都是不支持 win 路径分割的，得用 mingw 编译的 bin 才支持。。而 xrepo 仓库的 pkgconf 是支持的，所以一般优先用这个。

可以看下path 路径顺序。按理 add_deps("pkgconf") 后，路径应该会靠前，优先用才对

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

The bins of msys do not support win path segmentation. You must use the bin compiled by mingw to support it. . The pkgconf of the xrepo warehouse is supported, so this is generally used first.

You can look at the path sequence. Logically speaking, after adding_deps("pkgconf"), the path should be at the front, so it should be used first.

waruqi commented 1 month ago

可以调下这里，按理应该会优先用 pkgconf 的，前提是包配置了 add_deps pkgconf

https://github.com/xmake-io/xmake/blob/d342e78c2f24434935eb371d0dd5bba5410dbf3f/xmake/modules/package/tools/meson.lua#L54

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

You can adjust it here. It should be used first. pkgconf should be used first, provided that the package is configured with add_deps pkgconf.

https://github.com/xmake-io/xmake/blob/d342e78c2f24434935eb371d0dd5bba5410dbf3f/xmake/modules/package/tools/meson.lua#L54

star-hengxing commented 1 month ago

可以调下这里，按理应该会优先用 pkgconf 的，前提是包配置了 add_deps pkgconf

https://github.com/xmake-io/xmake/blob/d342e78c2f24434935eb371d0dd5bba5410dbf3f/xmake/modules/package/tools/meson.lua#L54

问题应该是这了，返回的是不是绝对路径，导致 meson 搜到 msys2 里的。

{
    "pkgconfig",
    "pkgconfig"
}

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

You can adjust this here, and pkgconf should be used first, provided that the package is configured with add_deps pkgconf

https://github.com/xmake-io/xmake/blob/d342e78c2f24434935eb371d0dd5bba5410dbf3f/xmake/modules/package/tools/meson.lua#L54

The problem should be this. Is the absolute path returned, causing meson to search for it in msys2?

{
    "pkgconfig",
    "pkgconfig"
}

waruqi commented 1 month ago

确定是 find_tool("pkgconf" 返回的 pkgconfig 不是下面那个 find_tool pkg-config 返回的？

find pkgconf 按理也不应该返回 pkgconfig 哈，即使没路径。。

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Are you sure that the pkgconfig returned by find_tool("pkgconf" is not the one returned by find_tool pkg-config below?

Find pkgconf should not return pkgconfig, even if there is no path. .

star-hengxing commented 1 month ago

确定是 find_tool("pkgconf" 返回的 pkgconfig 不是下面那个 find_tool pkg-config 返回的？

find pkgconf 按理也不应该返回 pkgconfig 哈，即使没路径。。

尴尬，这里手打错了。加了个 print + raise。

checking for pkgconf ... ok
{
  name = "pkgconf",
  program = "pkgconf"
}

error: stack traceback:
    [C]: in function 'error'
    [@programdir\core\base\os.lua:984]:
    [@programdir\modules\package\tools\meson.lua:57]: in function '_get_pkgconfig'
    [@programdir\modules\package\tools\meson.lua:463]: in function 'buildenvs'
    [@programdir\modules\package\tools\meson.lua:515]: in function 'generate'
    [@programdir\modules\package\tools\meson.lua:523]: in function 'build'
    [@programdir\modules\package\tools\meson.lua:545]: in function 'install'
    [A:\project\xmake\xmake-repo\packages\l\libvips\xmake.lua:134]: in function 'script'
    [...dir\modules\private\action\require\impl\utils\filter.lua:114]: in function 'call'
    [...\modules\private\action\require\impl\actions\install.lua:404]:

waruqi commented 1 month ago

msys2 里面也有个 pkgconf？另外有问题的是哪个包，那个包有没有加 add_deps("pkgconf") 配置呢？

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

There is also pkgconf in msys2? Also, which package is problematic? Is that package configured with add_deps("pkgconf")?

star-hengxing commented 1 month ago

msys2 里面也有个 pkgconf？

看最开始的问题描述。

Called: C:\Users\star\scoop\apps\msys2\2024-07-27\usr\bin\pkgconf.EXE --modversion glib-2.0 -> 1

最近装了 msys2 才发现有这些问题。

另外有问题的是哪个包，那个包有没有加 add_deps("pkgconf") 配置呢？

正在用 libvips 进行测试，目前有 deps 的 meson 包我都加的了

if is_plat("windows") then
    add_deps("pkgconf")
end

waruqi commented 1 month ago

Called: C:\Users\star\scoop\apps\msys2\2024-07-27\usr\bin\pkgconf.EXE --modversion glib-2.0 -> 1

看下 xmake.exe 装在哪，如果 pkgconf 包的 bin path 在 PATH 最开头，按理应该优先找到我们的，你可以 on_install 里 print 下 PATH 看下顺序。

目前我遇到过的只有一种情况，xmake.exe 和 msys2 pkgconf.exe 同在一个 bin 目录下，那么不管 PATH 怎么设置，都会优先找到 msys2 下的 pkgconf

https://github.com/xmake-io/xmake/issues/3628 类似这个

其他情况，那就不清楚了，按理即使返回非全路径 pkgconf 也应该优先用我们的。

正在用 libvips 进行测试，目前有 deps 的 meson 包我都加的了

meson 对 pkgconf 基本上算是必须的，我觉得挨个加也有点繁琐，可以把 add_deps("pkgconf") 搞进 meson 包里，

waruqi commented 1 month ago

C:\Users\star\scoop\apps\msys2\2024-07-27\usr\bin\pkgconf.EXE

它这个像是 meson 自己去找的，可能是 meson 的 pkgconfig= 设置，如果不给全路径，meson 就自己去找了？

xmake 的 find_tool -> find_program pkgconf 在 linux 上调用 which ，通常都会找到全路径。

而 win 上目前通过 where.exe 也能找到系统的全路径 pkgconf ，但是 xrepo 里面的 pkgconf ，就不一定了

不行就只能改进看看怎么让 find_pkgconf.lua 去返回全路径

star-hengxing commented 1 month ago

它这个像是 meson 自己去找的，可能是 meson 的 pkgconfig= 设置，如果不给全路径，meson 就自己去找了？

对，我硬编码 xrepo pkgconf 路径就没问题

而 win 上目前通过 where.exe 也能找到系统的全路径 pkgconf ，但是 xrepo 里面的 pkgconf ，就不一定了

不行就只能改进看看怎么让 find_pkgconf.lua 去返回全路径

这个简单，直接强制用 package:dep("pkgconf"):installdir("bin/pkgconf.exe") 保证稳定性

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

It seems that meson found it by itself. It may be the pkgconfig= setting of meson. If the full path is not given, meson will find it by itself?

Yes, I hardcoded the xrepo pkgconf path and it was fine.

On win, you can currently find the full path pkgconf of the system through where.exe, but the pkgconf in xrepo is not necessarily the same.

If it doesn’t work, we can only improve it and see how to make find_pkgconf.lua return the full path.

This is simple, just force package:dep("pkgconf"):installdir("bin/pkgconf.exe") to ensure stability

waruqi commented 1 month ago

试下这个 patch https://github.com/xmake-io/xmake/pull/5483

meson 包里 pkgconf 依赖内置不了，循环依赖了。

star-hengxing commented 1 month ago

试下这个 patch #5483

可以给 cmake 也应用这个修改，我两个构建系统测试了都没问题

Issues-translate-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically.

Try this patch #5483

You can also apply this modification to cmake. I tested both build systems and there was no problem.

xmake-io / xmake