CMake warnings about extra arguments relating to Python DLL import libraries in `build-llvm.sh`

[trilader]

I'm currently building a version of the llvm-mingw toolchain with some changed options and defaults (but I also tested this with a clean version of the repository).

While building the Windows native version of the toolchain (build-llvm.sh called from Dockerfile.cross) I noticed the following warnings show up (newlines were adjusted for readability):

CMake Warning:
  Ignoring extra path from command line:
   "/opt/llvm-mingw-x86_64/python/lib/libpython3.dll.a"

CMake Warning:
  Ignoring extra path from command line:
   "/opt/llvm-mingw-x86_64/python/lib/libpython3.dll.a"

After having a look at build-llvm.sh and finding the line thatresults in the messages:

PYTHON_LIB="$(echo $PREFIX/python/lib/libpython*.dll.a)"

and running the shell command in the temporary docker container that runs build-llvm.sh I got the following output:

docker run -it --rm <the-container-id> /bin/bash -c "echo /opt/llvm-mingw-x86_64/python/lib/libpython*.dll.a"
/opt/llvm-mingw-x86_64/python/lib/libpython3.11.dll.a /opt/llvm-mingw-x86_64/python/lib/libpython3.dll.a

There are two files matching the pattern and both get passed to the CMake commandline arguments. The warning message suggests CMake will be using libpython3.11.dll.a.

I also had a look at the symbols exported from both files using x86_64-w64-mingw32-objdump -t /opt/llvm-mingw-x86_64/python/lib/libpython3.dll.a | grep -v libpython3.dll | grep -v -e '^$' and noted that the 3.11.dll.a being used exports more symbols.

I'm going to assume that using the import library with more exports (which look like they relate to newer Python functions) is the correct behavior and the only thing "wrong" is the warning message which I could simply ignore?

[mstorsjo]

Thanks for noticing this, I had not seen this message. This seems to have been around since e7238335e75c9befe99c94b4b45144044cbd3946.

The reason for this seems to be https://github.com/msys2-contrib/cpython-mingw/commit/8aeb46d09c068a1228e83b29101c17bf1ff0612a - before this, the smaller DLL interface for a stable interface wasn't installed. (See https://peps.python.org/pep-0384/ for the original proposal about a stable Python ABI.) libpython3.dll has a smaller set of functions exposed, as it only exposes the bits that are part of the stable ABI.

Ideally I guess we should link against the stable ABI subset, but that doesn't seem to work, a build where the first path listed, which actually gets used by cmake, is libpython3.dll.a does fail. Are you sure that the messages you're getting, about the path that is ignored, is "/opt/llvm-mingw-x86_64/python/lib/libpython3.11.dll.a"? If it is ignoring that one, it means it's using libpython3.dll.a, which seems to fail to link LLDB for me?

So, indeed, you can ignore the warning message from CMake for now. Ideally we should improve the expression for setting PYTHON_LIB to only pick the versioned one.

[trilader]

Are you sure that the messages you're getting, about the path that is ignored, is "/opt/llvm-mingw-x86_64/python/lib/libpython3.11.dll.a"?

That was indeed in the content of my report initially, until I noticed I copied the output from the wrong build session logs. I've since then updated my report.

I've also observed the linking of LLDB failing in that case, which was the thing that got me to notice that in the first place.

I noticed this whole thing in the first place because I'm currently experimenting with/working on a patch to allow the Python installation (and especially the libpython.dll to have a suffix (so getting build and installed as libpython-llvm.dll) as to not collide with another Python installation also on $PATH in combination with not that great build scripts for other software.

[mstorsjo]

Are you sure that the messages you're getting, about the path that is ignored, is "/opt/llvm-mingw-x86_64/python/lib/libpython3.11.dll.a"?

That was indeed in the content of my report initially, until I noticed I copied the output from the wrong build session logs. I've since then updated my report.

Ok, thanks for confirming!

I've also observed the linking of LLDB failing in that case, which was the thing that got me to notice that in the first place.

Ok, good to know that we're seeing the same there.

I noticed this whole thing in the first place because I'm currently experimenting with/working on a patch to allow the Python installation (and especially the libpython.dll to have a suffix (so getting build and installed as libpython-llvm.dll) as to not collide with another Python installation also on $PATH in combination with not that great build scripts for other software.

Ah, I see!

FWIW, I made an attempt at a fix for this, see https://github.com/mstorsjo/llvm-mingw/commit/c44e4321533c11ad40ff5067cb7c4cca263a9235. When the CI run at https://github.com/mstorsjo/llvm-mingw/actions/runs/7710531683 finishes, I'll land this on the master branch.

[mstorsjo]

Fixed in https://github.com/mstorsjo/llvm-mingw/commit/cb52ece4112f72591a37f8f3b7cd79a63a0d2c3f.