[gdal] build failure

[insar-dev]

hi, I installed gdal successfully in other machine, but failed in new machine. it seems that something wrong in include path.

cc1plus: warning: /home/cuhk/vcpkg/installed/x64-linux/debug/lib/pkgconfig/../../../../include: No such file or directory [-Wmissing-include-dirs]

it does not exist: /home/cuhk/vcpkg/installed/x64-linux/debug/lib/pkgconfig/../../../../include it exists : /home/cuhk/vcpkg/installed/x64-linux/debug/lib/pkgconfig/../../../include

Host Environment

• OS: ubuntu 20.04
• Compiler: gcc 11

To Reproduce Steps to reproduce the behavior: ./vcpkg install gdal

Failure logs -Cut and paste the appropriate build messages from the console output. -Please attach any additional failure logs mentioned in the console output.

Computing installation plan...
The following packages will be built and installed:
    gdal[core,hdf5,netcdf,postgresql,supported-default-features]:x64-linux -> 3.4.1#1
Detecting compiler hash for triplet x64-linux...
Restored 0 packages from /home/cuhk/.cache/vcpkg/archives in 5.379 us. Use --debug to see more details.
Starting package 1/1: gdal:x64-linux
Building package gdal[core,hdf5,netcdf,postgresql,supported-default-features]:x64-linux...
-- Using cached OSGeo-gdal-v3.4.1.tar.gz.
-- Cleaning sources at /home/cuhk/vcpkg/buildtrees/gdal/src/v3.4.1-8905a3e6e8.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source /home/cuhk/vcpkg/downloads/OSGeo-gdal-v3.4.1.tar.gz
-- Applying patch 0001-Fix-debug-crt-flags.patch
-- Applying patch 0002-Fix-build.patch
-- Applying patch 0004-Fix-cfitsio.patch
-- Applying patch 0005-Fix-configure.patch
-- Applying patch 0007-Control-tools.patch
-- Applying patch 0008-Fix-absl-string_view.patch
-- Using source at /home/cuhk/vcpkg/buildtrees/gdal/src/v3.4.1-8905a3e6e8.clean
-- Getting CMake variables for x64-linux-dbg
-- Getting CMake variables for x64-linux-rel
-- Generating configure for x64-linux
-- Finished generating configure for x64-linux
-- Configuring x64-linux-dbg
-- Configuring x64-linux-rel
-- Building x64-linux-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:158 (message):
    Command failed: /usr/bin/make V=1 -j 9 -f GNUmakefile all
    Working Directory: /home/cuhk/vcpkg/buildtrees/gdal/x64-linux-dbg/
    See logs for more information:
      /home/cuhk/vcpkg/buildtrees/gdal/build-x64-linux-dbg-out.log
      /home/cuhk/vcpkg/buildtrees/gdal/build-x64-linux-dbg-err.log

Call Stack (most recent call first):
  scripts/cmake/vcpkg_build_make.cmake:187 (vcpkg_execute_build_process)
  scripts/cmake/vcpkg_install_make.cmake:26 (vcpkg_build_make)
  ports/gdal/portfile.cmake:317 (vcpkg_install_make)
  scripts/ports.cmake:145 (include)

Error: Building package gdal:x64-linux failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `git pull` and `./vcpkg update`.
Then check for known issues at:
  https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+gdal
You can submit a new issue at:
  https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=[gdal]+Build+error
including:
  package: gdal[core,hdf5,netcdf,postgresql,supported-default-features]:x64-linux -> 3.4.1#1
    vcpkg-tool version: 2022-02-01-c76e0def147898591d9c35113b95e0b3f2eaa4d4
    vcpkg-scripts version: ce7ee5ac3 2022-02-04 (8 hours ago)

Additionally, attach any relevant sections from the log files above.

build-x64-linux-dbg-err.log build-x64-linux-dbg-out.log

Additional context Add any other context about the problem here, such as what you have already tried to resolve the issue.

[dg0yt]

Can you provide config.log...? Did you build from a clean install tree? The gdal build relies on proper pkg-config data. You might try to grep for ../../../.. in debug/lib/pkgconfig/*.pc. If you have a hit, remove the offending package.

[insar-dev]

@dg0yt thank you for your reply.

config-x64-linux-dbg-err.log config-x64-linux-dbg-out.log

I build from a clean install tree, so there is no gdal.pc in pkgconfig.

[dg0yt]

I really mean the config.log......log files. This is the detailed log from configure.

[insar-dev]

@dg0yt there is no config.log file in buildtrees/gdal. could you tell me where to find it ?

[dg0yt]

Originally, it is created as config.log in each build directory (x64-libux-dbg etc.).

[insar-dev]

@dg0yt

[dg0yt]

... because it is moved after successful configure. So look again for config.log-x64-linux-dbg.log next to the other logs.

[insar-dev]

i found the config.log-x64-linux-dbg.log file @dg0yt config.log-x64-linux-dbg.log

[dg0yt]

It is strange, all paths from pkg-config data carry an extra ../, e.g.

EXPAT_CFLAGS='-I/home/cuhk/vcpkg/installed/x64-linux/debug/lib/pkgconfig/../../../../include'
EXPAT_INCLUDE='-I/home/cuhk/vcpkg/installed/x64-linux/debug/lib/pkgconfig/../../../../include'
EXPAT_LIBS='-L/home/cuhk/vcpkg/installed/x64-linux/debug/lib/pkgconfig/../../../lib -lexpat'

Can you please also share installed/x64-linux/debug/lib/expat.pc? This is mine from a fresh build, and it looks valid:

prefix=${pcfiledir}/../..

exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/../include

Name: expat
Version: 2.4.1
Description: expat XML parser
URL: http://www.libexpat.org
Libs: -L"${libdir}" -lexpat
Cflags: -I"${includedir}"

Are there any environment variables set that modify PKG_CONFIG behaviours? (env | grep PKG_CONFIG)?

[insar-dev]

expat.pc: prefix=${pcfiledir}/../../..

exec_prefix=${prefix} libdir=${exec_prefix}/lib includedir=${prefix}/../include

Name: expat Version: 2.4.1 Description: expat XML parser URL: http://www.libexpat.org Libs: -L"${libdir}" -lexpat Cflags: -I"${includedir}"

@dg0yt No environment variable modify PKG_CONFIG behaviour.

[insar-dev]

I just reinstalled vckpg, and then installed gdal, it has the same problem.

[dg0yt]

Hm, where does the wrong prefix come from? The prefix= line is written by vcpkg_fixup_pkg_config at after the actual build and installation: https://github.com/microsoft/vcpkg/blob/7e7dad5fe20cdc085731343e0e197a7ae655555b/scripts/cmake/vcpkg_fixup_pkgconfig.cmake#L194 (Before fixup, it is an absolute path. You can see it in the build directory.)

Note: this is not an gdal issue. It is an issue with pc files.

[insar-dev]

which part would produce the wrong path? how can i to debug ? it is strange that my another computer is ok, they have same system and develop environment.

[insar-dev]

@sfhacker expat was installed automatically by vcpkg. expat is one of gdal dependencies.
i found all gdal dependencies pc file have same issue.

GEOS.pc prefix=${pcfiledir}/../../..

exec_prefix=${prefix} includedir=${prefix}/../include libdir=${exec_prefix}/lib

Name: GEOS Description: Geometry Engine, Open Source - C API Version: 3.10.0 Libs: -L"${libdir}" -lgeos_cd -lgeosd -lstdc++ -lm Requires: Cflags: -I"${includedir}"

libpng.pc: prefix=${pcfiledir}/../../..

exec_prefix=${prefix} libdir=${prefix}/lib includedir=${prefix}/../include/libpng16

Name: libpng Description: Loads and saves PNG files Version: 1.6.37 Libs: -L"${libdir}" -lpng16d -lz -lm Requires: zlib Cflags: -I"${includedir}"

[JackBoosY]

Where have I seen similar issues...

[dg0yt]

The 'prefix=' key or variable should contain an absolute path!

prefix=${pcfiledir}/../../.. is an absolute path.

How did you install expat?

expat was installed automatically by vcpkg. i found all gdal dependencies pc file have same issue.

This was visible from config.log.

which part would produce the wrong path? how can i to debug ?

If you do know cmake, you can instrument the section in vcpkg_fixup_pkg_config to trace how the new prefix is derived.

[insar-dev]

i manually modified pc files. And now vcpkg can install gdal normally.

[dg0yt]

It may occur again as soon as you restart from a fresh tree, or pull updates from vcpkg. So far, it seems to be not reproducable for anyone else.

[insar-dev]

I have tried to solve the issue, but failed. Could someone give some more detailed suggestions?

[dg0yt]

Could someone give some more detailed suggestions?

I suggested this: If you do know cmake, you can instrument the section in vcpkg_fixup_pkg_config to trace how the new prefix is derived.

If you want more details, you should indicate which prior knowledge we can assume.

[insar-dev]

i am not familiar with cmake, i will try my best to do it.

[insar-dev]

hi, i added some message to track issue. finally find that relative_pc_path value is:../../..

relative_pc_path:/home/cuhk/vcpkg/packages/expat_x64-linux/debug pkg_lib_search_path:/home/cuhk/vcpkg/packages/expat_x64-linux/debug/lib/pkgconfig/expat.pc modified relative_pc_path: ../../..

[dg0yt]

Could you please also print ${file} right after foreach?

I expect file to be /home/cuhk/vcpkg/packages/expat_x64-linux/debug/lib/pkgconfig/expat.pc, but then we should get pkg_lib_search_path as /home/cuhk/vcpkg/packages/expat_x64-linux/debug/lib/pkgconfig which it isn't according to your report.

[insar-dev]

file is :/home/cuhk/vcpkg/packages/expat_x64-linux/debug/lib/pkgconfig/expat.pc

[JackBoosY]

Can you please clone a new vcpkg and try again?

[dg0yt]

So it comes down to an issue with cmake_path(GET file PARENT_PATH pkg_lib_search_path). This is pure CMake, and the wrong result is only on your computer. It is a fairly new command, but I didn't find any issue reports about it.

For reference, I already tested this script with CMake 3.21 on my Ubuntu 18.04 computer:

message(STATUS "CMake ${CMAKE_VERSION}")
set(file /home/cuhk/vcpkg/packages/expat_x64-linux/debug/lib/pkgconfig/expat.pc)
message(STATUS "file:                ${file}")
cmake_path(GET file PARENT_PATH pkg_lib_search_path)
message(STATUS "pkg_lib_search_path: ${pkg_lib_search_path}")

from my vcpkg dir with:

$ downloads/tools/cmake-3.21.1-linux/cmake-3.21.1-linux-x86_64/bin/cmake -P test.cmake 
-- CMake 3.21.1
-- file:                /home/cuhk/vcpkg/packages/expat_x64-linux/debug/lib/pkgconfig/expat.pc
-- pkg_lib_search_path: /home/cuhk/vcpkg/packages/expat_x64-linux/debug/lib/pkgconfig

[insar-dev]

Can you please clone a new vcpkg and try again?

@JackBoosY I have done it yesterday.

[insar-dev]

@dg0yt i will try to reinstall cmake to 3.21.

[dg0yt]

Are you targeting x64-linux from Windows? Is this WSL?

[insar-dev]

@dg0yt yes, not WSL I reinstalled vcpkg and reinstalled cmake to 3.21.1, vckpkg can install gdal successfully now. There is a bug in cmake 3.22.2.

[JackBoosY]

Can anyone report this bug to cmake https://gitlab.kitware.com/cmake/cmake/-/issues?

[insar-dev]

let me report this bug to cmake.

[dg0yt]

It is important to include the environment into the CMake bug report. This function is unit-tested.

[insar-dev]

I reported the bug to cmake. https://gitlab.kitware.com/cmake/cmake/-/issues/23187

[insar-dev]

I have changed cmake to 3.21.1, then there is another issue: cmake can not find some libraries that build by vcpkg(using cmake 3.22.2)

How to solve it? Thanks for your suggestion.

The same project in linux vscode, there is no issure, but has some warnings:

[dg0yt]

I don't see anyhing related to gdal here. In any case, you should get rid of the cached artifacts which were built with the broken cmake.

[insar-dev]

I will try it . Thanks.

[JackBoosY]

@insar-dev Can you please follow the cmake mantainer's suggestion and provide the output? I can't repro this issue using cmake 3.22.2 locally.

Thanks.

[JackBoosY]

@insar-dev The python-pip side has been fixed this, I think you should re-install cmake using pip now.