Python bindings should prefer bundled GDAL over system packages

[clickworkorange]

What is the bug?

On completing a fresh build of GDAL 3.9.2 the Python bindings fail to build with

  File "/myprojectdir/gdal-3.9.2/build/swig/python/setup.py", line 276, in finalize_options
    raise Exception(f"Python bindings of GDAL {version} require at least libgdal {version}, but {library_version} was found")
Exception: Python bindings of GDAL 3.9.2 require at least libgdal 3.9.2, but 3.6.2 was found

Unfortunately GDAL 3.6.2 is required by a bunch of things on my system, including Blender. It would be nice if the GDAL Python bindings build script would look in the build directory for GDAL before trying the system libs; GDAL 3.9.2 is sitting right there.

Steps to reproduce the issue

Download and unpack gdal-3.9.2.tar.gz. Then:

$ cd gdal-3.9.2
$ mkdir build && cd build
$ cmake ..
$ cmake --build .
$ cmake --build . --target install

Versions and provenance

Devuan Daedalus Linux 6.1.0-25-amd64 cmake 3.25.1 Python 3.11.2 gcc/g++ 12.2.0

Additional context

No response

[rouault]

In your build environment, does gdal-config --version returns something ?

[clickworkorange]

Yep, 3.9.2:

/myprojectdir/gdal-3.9.2/build$ gdal-config --version
3.9.2

[rouault]

@clickworkorange hum I'm a bit at loss then where it finds your other GDAL version ? Where is 3.6.2 is installed ?

[clickworkorange]

It's a dependency of Blender and Flowblade, amongst other things:

$ dpkg --status libgdal32
Package: libgdal32
Status: install ok installed
Priority: optional
Section: libs
Installed-Size: 24026
Maintainer: Debian GIS Project <pkg-grass-devel@lists.alioth.debian.org>
Architecture: amd64
Source: gdal (3.6.2+dfsg-1)
Version: 3.6.2+dfsg-1+b2
...

$ sudo apt remove libgdal32
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
  blender-data gdal-data gdal-plugins hdf5-helpers hyphen-en-us icc-profiles-free libarmadillo-dev libarmadillo11 libarpack2-dev
  libblosc-dev libblosc1 libboost-all-dev libboost-atomic-dev libboost-chrono-dev libboost-container-dev libboost-context-dev
  libboost-coroutine-dev libboost-exception-dev libboost-fiber-dev libboost-graph-parallel-dev libboost-locale-dev libboost-log-dev
  libboost-math-dev libboost-mpi-dev libboost-mpi-python-dev libboost-nowide-dev libboost-numpy-dev libboost-random-dev
  libboost-stacktrace-dev libboost-system-dev libboost-test-dev libboost-timer-dev libboost-tools-dev libboost-type-erasure-dev
  libboost-wave-dev libcfitsio-dev libcfitsio-doc libcharls2 libcollada-dom2.5-dp0 libdav1d-dev libdc1394-dev libdcmtk17
  libde265-dev libembree3-3 libflann-dev libflann1.9 libfreexl-dev libfreexl1 libfyba-dev libfyba0 libgdcm-dev libgdcm3.0
  libgeos-c1v5 libgeos-dev libgeos3.11.1 libgeotiff-dev libgeotiff5 libgif-dev libgl2ps-dev libglut3.12 libgphoto2-dev
  libgraphicsmagick++-q16-12 libgraphicsmagick-q16-3 libhdf4-0-alt libhdf4-alt-dev libhdf5-cpp-103-1 libhdf5-dev libhdf5-fortran-102
  libhdf5-hl-cpp-100 libhdf5-hl-fortran-100 libheif-dev libjson-c-dev libkml-dev libkmlbase1 libkmlconvenience1 libkmldom1
  libkmlengine1 libkmlregionator1 libkmlxsd1 liblog4cplus-2.0.5 libminizip-dev libmlt-data libmovit8 libnetcdf-c++4
  libnetcdf-cxx-legacy-dev libnetcdf-dev libogdi-dev libogdi4.1 libopencolorio2.1 libopencv-core-dev libopencv-dnn-dev
  libopencv-flann-dev libopencv-highgui406 libopencv-imgproc-dev libopencv-ml-dev libopencv-ml406 libopencv-photo-dev
  libopencv-photo406 libopencv-shape-dev libopencv-shape406 libopencv-stitching406 libopencv-video-dev libopencv-viz-dev
  libopencv-viz406 libopenni-dev libopenni-sensor-pointclouds0 libopenni0 libopenni2-dev libopenthreads21 libopenvdb10.0
  libosdcpu3.5.0 libosdgpu3.5.0 libpcl-apps1.13 libpcl-common1.13 libpcl-features1.13 libpcl-filters1.13 libpcl-io1.13
  libpcl-kdtree1.13 libpcl-keypoints1.13 libpcl-ml1.13 libpcl-octree1.13 libpcl-outofcore1.13 libpcl-people1.13
  libpcl-recognition1.13 libpcl-registration1.13 libpcl-sample-consensus1.13 libpcl-search1.13 libpcl-segmentation1.13
  libpcl-stereo1.13 libpcl-surface1.13 libpcl-tracking1.13 libpcl-visualization1.13 libpodofo0.9.8 libpoppler-dev
  libpoppler-private-dev libpystring0 libqhull-dev libqhull8.0 libqhullcpp8.0 libqt5webkit5-dev libraw1394-dev libraw1394-tools
  librtaudio6 librttopo-dev librttopo1 libsocket++1 libsox-fmt-alsa libsox-fmt-base libsox3 libspatialite-dev libspatialite7
  libsuperlu-dev libsuperlu5 libtbb-dev libtheora-dev libtinyxml2.6.2v5 liburiparser-dev liburiparser1 libusb-1.0-0-dev
  libusb-1.0-doc libutfcpp-dev libvtk9-java libvtk9.1-qt opencv-data openni-utils python3-mpi4py python3-vtk9 qttools5-private-dev
  scribus-data tcl-dev tcl8.6-dev tk-dev tk8.6-dev vtk9
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  blender flowblade gmic libgdal-dev libgdal32 libgmic1 libmlt++7 libmlt7 libopencv-calib3d-dev libopencv-contrib-dev
  libopencv-contrib406 libopencv-dev libopencv-features2d-dev libopencv-highgui-dev libopencv-imgcodecs-dev libopencv-imgcodecs406
  libopencv-java libopencv-objdetect-dev libopencv-stitching-dev libopencv-superres-dev libopencv-superres406 libopencv-videoio-dev
  libopencv-videoio406 libopencv-videostab-dev libopencv-videostab406 libopencv406-jni libopenimageio2.4 libopenscenegraph161
  libpcl-dev libvtk9-dev libvtk9-qt-dev python3-mlt python3-opencv scribus
0 upgraded, 0 newly installed, 34 to remove and 21 not upgraded.
After this operation, 482 MB disk space will be freed.
Do you want to continue? [Y/n] absolutely not

[clickworkorange]

Maybe that was not the answer you were looking for. On my system libgdal 3.6.2 is found here: /usr/lib/x86_64-linux-gnu/libgdal.so.32.3.6.2. It was installed there by apt, as a dependency of some of the applications I use (or a dependency of a dependency, etc). As shown above I can't really uninstall it. I'm hoping there's still a way to compile 3.9.2 from source, because I need to include it in a project I'm tinkering with.

[clickworkorange]

It seems this issue has been reported before, but neither the fix or the work-around prevent it happening for me: https://github.com/OSGeo/gdal/issues/8882

[clickworkorange]

Finally managed to get GDAL 3.9.2 to build by manually editing /myprojectdir/gdal-3.9.2/build/swig/python/setup.py and pointing it directly to the 3.9.2 gdal-config:

    def initialize_options(self):
...
        #self.gdal_config = self.GDAL_CONFIG
        self.gdal_config = "/myprojectdir/gdal-3.9.2/build/apps/gdal-config" 
...

I have no idea if this hacky fix caused any issues, but GDAL seemed to build without error.