search

rpm-software-management / libcomps

Libcomps is alternative for yum.comps library. It's written in pure C as library and there's bindings for python2 and python3.
GNU General Public License v2.0
29 stars 37 forks source link

Fix building for Python 3.9 #64

Closed m-blaha closed 4 years ago

hroncok commented 4 years ago

Applying this patch to the Fedora package makes it build with 3.9. (However I have no idea what else does it potentially break, so I cannot review it.)

hroncok commented 4 years ago

is this relevant?

https://github.com/Kitware/CMake/blob/master/Modules/FindPythonLibs.cmake#L82

lukash commented 4 years ago

is this relevant?

https://github.com/Kitware/CMake/blob/master/Modules/FindPythonLibs.cmake#L82

Nice find! How this is supposed to be future-proof, I wonder?

hroncok commented 4 years ago

How this is supposed to be future-proof, I wonder?

You just poke it with a stick every once in a while https://gitlab.kitware.com/cmake/cmake/merge_requests/4225

Seriously, Fedora packages should either use /usr/bin/python3 or get an explicit argument of Python to use. Any "autodetection" like this is very fragile.

lukash commented 4 years ago

How this is supposed to be future-proof, I wonder?

You just poke it with a stick every once in a while https://gitlab.kitware.com/cmake/cmake/merge_requests/4225

I don't want to derail, but are you supposed to update to bleeding-edge CMake to be able to build against newest Python? In addition to someone having to file a bug with them to actually add the support? That just doesn't look to be a good solution.

Seriously, Fedora packages should either use /usr/bin/python3 or get an explicit argument of Python to use. Any "autodetection" like this is very fragile.

What exactly do you mean by auto-detection? The piece of code in our CMake (that really doesn't do much) or the CMake macros we use?

It's not only about /usr/bin/python3 but mainly about linking against python, which requires build flags and that's mainly what the CMake macros do. We need something like that and if our software is compatible with say all versions of python 3, you just wanna use the python 3 version you find on the system and not require it being passed to you (that's how linking works in general). I'm not too sure about what exactly the CMake macros do though.

Or maybe I'm misunderstanding your point?

hroncok commented 4 years ago

I don't want to derail, but are you supposed to update to bleeding-edge CMake to be able to build against newest Python?

That's why I think that the FindPython cmake thing si wrong.

What exactly do you mean by auto-detection?

When you use FindPython, cmake tries to automatically detect the best Python version for you. It does it by having a hardcoded list of possible versions. Both is IMHO wrong.

It's not only about /usr/bin/python3 but mainly about linking against python, which requires build flags and that's mainly what the CMake macros do.

Python has a way to provide the flags to you. You can either use distutils/setptools or python3-config.

you just wanna use the python 3 version you find on the system and not require it being passed to you

Yes. You use /usr/bin/python3 by default, or the passed version. With cmake, it uses the highest available Python version that cmake knows. I think that's a bad default.

My point is that what FindPython does is wrong on so many levels.

lukash commented 4 years ago

I don't want to derail, but are you supposed to update to bleeding-edge CMake to be able to build against newest Python?

That's why I think that the FindPython cmake thing si wrong.

I'll agree on that :slightly_smiling_face:

When you use FindPython, cmake tries to automatically detect the best Python version for you. It does it by having a hardcoded list of possible versions. Both is IMHO wrong.

I find the hardcoded version list to be a bad concept, but... there are more use-cases in slightly different environments (mostly different distros) which I don't know about or fully understand.

Like.. in general you can have multiple versions of Python installed on the system, even minor versions, like 3.6, 3.7 and 3.8. But then on such a system the default version of Python 3 to build against should be specified by the system - at least on Gentoo it works this way. And it's similar to how on e.g. Debian it is common to have multiple ABI versions of a native library (differentiated by a version in the package name, e.g. libfoo2-2.1 and libfoo3-3.0) and then you have a single -dev package installed (e.g. libfoo-dev-2.1) which means you're building against the version 2.

This doesn't seem to be common on RPM based distros and when needed, it seems to be done in a non-standardized, ad-hoc way (at least that's my impression so far).

So anyway, can you have multiple (minor?) versions of Python on Fedora and if yes, what says which is the version to build against?

Python has a way to provide the flags to you. You can either use distutils/setptools or python3-config.

distutils/setptools is for when using the native Python setup.py build system AFAIK.

Not sure what python3-config exactly is, but if it's a generic tool, it can and probably should be wrapped by a CMake macro (maybe that's what's used by the current macros under the hood?), so that the boilerplate for this very common piece is part of CMake itself. Such a wrapper shouldn't introduce hardcoded versions :slightly_smiling_face:

Btw. I'm continuing this discussion to try and form an opinion on how to build against Python in our CMake scripts, as we have multiple variants across the components and not a clear way to approach it. Until we get it right, there are bound to be problems...

hroncok commented 4 years ago

So anyway, can you have multiple (minor?) versions of Python on Fedora and if yes, what says which is the version to build against?

Yes. You.

lukash commented 4 years ago

So anyway, can you have multiple (minor?) versions of Python on Fedora and if yes, what says which is the version to build against?

Yes. You.

Well, that may be the problem. I think there are instances where you don't want to be specifying against which Python to build during the build process, you want it specified in the environment. At least that's what I think. It's for those cases that CMake has this autodetection you don't like. (As I said, on other distros I have experience with it is part of the system setup. That's what I'm basing my opinion on for the most part.)

hroncok commented 4 years ago

you want it specified in the environment

That's why the build machinery should default to /usr/bin/python3 and allow custom explciit Python override.

Anyway, I'm, not fully dedicated to this discussion. I think we both agree that what cmake does here is wrong, and we may dislike that, but that's all I'm going to do, I'm certainly not investing my time into making cmake better.

I guess what I wanted to say is that the Fedora package for libcomps should do something like %cmake ... PYTHON=%{python3_version} ... to bypass this autodetection altogether. Yet cmake might still fail if it doesn't know this version yet, so it might not sove anything.

I'll gladly finish this conversation over :coffee: or :beers:

lukash commented 4 years ago

No worries. I'm mindful about spending my time too and I also don't wanna go off fixing CMake. I'm just thinking we should first know what behavior we want and then see how to achieve it with what CMake has in place.

That's why the build machinery should default to /usr/bin/python3 and allow custom explciit Python override.

You keep mentioning /usr/bin/python3, but IMHO that's not what this is about, it's about linking against Python's .sos in the right version. And there may be more versions installed, so unless /usr/bin/python3 tells me which version to pick (which would seem hackish to me), we're back to autodetection.

Anyways, thanks for the info and next time this comes up I'll try to figure out something.

I'll gladly finish this conversation over coffee or beers

Might as well do that over at devconf if we get the chance :slightly_smiling_face:

hroncok commented 4 years ago

Oh, that's what I actually mean /usr/bin/python3 will tell you what to link with:

$ python3 -m sysconfig
...

(Ugly, but it is there.) Or even:

$ python3-config --libs
-lpython3.7m -lcrypt -lpthread -ldl  -lutil -lm

Might as well do that over at devconf if we get the chance

Sure. I'll try to be visible.

evgeni commented 4 years ago

FWIW, this also breaks on EL7 with python3 :(

/builddir/build/BUILD/libcomps-libcomps-0.1.15/libcomps/src/python
Building for python3
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:108 (message):
  Could NOT find PythonInterp: Found unsuitable version "3.6.8", but required
  is exact version "3" (found /usr/bin/python3)
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:313 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake/Modules/FindPythonInterp.cmake:139 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  src/python/src/CMakeLists.txt:49 (find_package)
-- Configuring incomplete, errors occurred!
See also "/builddir/build/BUILD/libcomps-libcomps-0.1.15/build-py3/CMakeFiles/CMakeOutput.log".

Dropping the EXACT helps.