conda-forge / gdal-feedstock

A conda-smithy repository for gdal.
BSD 3-Clause "New" or "Revised" License
30 stars 63 forks source link

Linking problem on linux-64: undefined reference to `memcpy@GLIBC_2.14' #777

Open leonid-butenko opened 1 year ago

leonid-butenko commented 1 year ago

Solution to issue cannot be found in the documentation.

Issue

Cannot link to libgdal. Any ideas?

$> docker run -it --rm -v /work:/work condaforge/miniforge3
$> conda install -y gcc_linux-64  gxx_linux-64 libgdal
$> $GXX -I/opt/conda/include -L/opt/conda/lib -lgdal test_gdal.cpp -o test_gdal

test_gdal.cpp:43:25: warning: ignoring return value of 'CPLErr GDALRasterBand::RasterIO(GDALRWFlag, int, int, int, int, void*, int, int, GDALDataType, GSpacing, GSpacing, GDALRasterIOExtraArg*)' declared with attribute 'warn_unused_result' [-Wunused-result]
   43 |         poBand->RasterIO(GF_Write, 0, 0, nXSize, nYSize, flatData.data(), nXSize, nYSize, GDT_Float64, 0, 0);
      |         ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_gdal.cpp: In member function 'double Interpolator2D::interpolate(double, double)':
test_gdal.cpp:52:28: warning: ignoring return value of 'int GDALInvGeoTransform(double*, double*)' declared with attribute 'warn_unused_result' [-Wunused-result]
   52 |         GDALInvGeoTransform(adfGeoTransform, inv_geotransform);
      |         ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_gdal.cpp:66:25: warning: ignoring return value of 'CPLErr GDALRasterBand::RasterIO(GDALRWFlag, int, int, int, int, void*, int, int, GDALDataType, GSpacing, GSpacing, GDALRasterIOExtraArg*)' declared with attribute 'warn_unused_result' [-Wunused-result]
   66 |         poBand->RasterIO(GF_Read, x, y, 2, 2, adfPixel, 2, 2, GDT_Float64, 0, 0);
      |         ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/13.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: /opt/conda/lib/libgdal.so: undefined reference to `posix_spawnp@GLIBC_2.15'
/opt/conda/bin/../lib/gcc/x86_64-conda-linux-gnu/13.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: /opt/conda/lib/libgdal.so: undefined reference to `memcpy@GLIBC_2.14'
collect2: error: ld returned 1 exit status

Installed packages

# packages in environment at /opt/conda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
binutils_impl_linux-64    2.40                 hf600244_0    conda-forge
binutils_linux-64         2.40                 hbdbef99_0    conda-forge
blosc                     1.21.4               h0f2a231_0    conda-forge
boltons                   23.0.0             pyhd8ed1ab_0    conda-forge
boost-cpp                 1.78.0               h6582d0a_3    conda-forge
brotli-python             1.0.9           py310hd8f1fbe_9    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.19.1               hd590300_0    conda-forge
ca-certificates           2023.5.7             hbcca054_0    conda-forge
cairo                     1.16.0            hbbf8b49_1016    conda-forge
certifi                   2023.5.7           pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310h255011f_3    conda-forge
cfitsio                   4.2.0                hd9d235c_0    conda-forge
charset-normalizer        3.1.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     23.5.0          py310hff52083_1    conda-forge
conda-package-handling    2.0.2              pyh38be061_0    conda-forge
conda-package-streaming   0.8.0              pyhd8ed1ab_0    conda-forge
cryptography              41.0.1          py310h75e40e8_0    conda-forge
curl                      8.1.2                h409715c_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               hca18f0e_1    conda-forge
freexl                    1.0.6                h166bdaf_1    conda-forge
gcc_impl_linux-64         13.1.0               hc4be1a9_0    conda-forge
gcc_linux-64              13.1.0               hd2d1137_0    conda-forge
geos                      3.12.0               h59595ed_0    conda-forge
geotiff                   1.7.1                h22adcc9_9    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
gxx_impl_linux-64         13.1.0               hc4be1a9_0    conda-forge
gxx_linux-64              13.1.0               h5006d2a_0    conda-forge
hdf4                      4.2.15               h501b40f_6    conda-forge
hdf5                      1.14.1          nompi_h4f84152_100    conda-forge
icu                       72.1                 hcb278e6_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
json-c                    0.16                 hc379101_0    conda-forge
jsonpatch                 1.32               pyhd8ed1ab_0    conda-forge
jsonpointer               2.0                        py_0    conda-forge
kealib                    1.5.1                h3e6883b_4    conda-forge
kernel-headers_linux-64   2.6.32              he073ed8_15    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.20.1               h81ceb04_0    conda-forge
lcms2                     2.15                 haa2dc70_1    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.0.6                hcb278e6_1    conda-forge
libarchive                3.6.2                h039dbb9_1    conda-forge
libcurl                   8.1.2                h409715c_0    conda-forge
libdeflate                1.18                 h0b41bf4_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     13.1.0               he3cc6c4_0    conda-forge
libgcc-ng                 13.1.0               he5830b7_0    conda-forge
libgdal                   3.7.0                h5a391c5_4    conda-forge
libgfortran-ng            13.1.0               h69a702a_0    conda-forge
libgfortran5              13.1.0               h15d22d2_0    conda-forge
libglib                   2.76.4               hebfc3b9_0    conda-forge
libgomp                   13.1.0               he5830b7_0    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             2.1.5.1              h0b41bf4_0    conda-forge
libkml                    1.3.0             h37653c0_1015    conda-forge
libnetcdf                 4.9.2           nompi_he09a3a9_107    conda-forge
libnghttp2                1.52.0               h61bc06f_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libpq                     15.3                 hbcd7760_1    conda-forge
librttopo                 1.1.0               hb58d41b_14    conda-forge
libsanitizer              13.1.0               hfd8a6a1_0    conda-forge
libspatialite             5.0.1               h15f6e67_28    conda-forge
libsqlite                 3.42.0               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-devel_linux-64  13.1.0               he3cc6c4_0    conda-forge
libstdcxx-ng              13.1.0               hfd8a6a1_0    conda-forge
libtiff                   4.5.1                h8b53f26_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.1                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.11.4               h0d562d8_0    conda-forge
libzip                    1.9.2                hc929e4a_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
ncurses                   6.4                  hcb278e6_0    conda-forge
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.89                 he45b914_0    conda-forge
openjpeg                  2.5.0                hfec8fc6_2    conda-forge
openssl                   3.1.1                hd590300_1    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pip                       23.1.2             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
pluggy                    1.2.0              pyhd8ed1ab_0    conda-forge
poppler                   23.05.0              hd18248d_1    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
postgresql                15.3                 hd458b1d_1    conda-forge
proj                      9.2.1                ha643af7_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pycosat                   0.6.4           py310h5764c6d_1    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 23.2.0             pyhd8ed1ab_1    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.10.12         hd12c33a_0_cpython    conda-forge
python_abi                3.10                    3_cp310    conda-forge
readline                  8.2                  h8228510_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
ruamel.yaml               0.17.32         py310h2372a71_0    conda-forge
ruamel.yaml.clib          0.2.7           py310h1fa729e_1    conda-forge
setuptools                68.0.0             pyhd8ed1ab_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
sqlite                    3.42.0               h2c6b66d_0    conda-forge
sysroot_linux-64          2.12                he073ed8_15    conda-forge
tiledb                    2.13.2               hd532e3d_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tqdm                      4.65.0             pyhd8ed1ab_1    conda-forge
tzcode                    2023c                h0b41bf4_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
urllib3                   2.0.3              pyhd8ed1ab_1    conda-forge
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
xerces-c                  3.2.4                h8d71039_2    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.6                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstandard                 0.19.0          py310h1275a96_2    conda-forge
zstd                      1.5.2                h3eb15da_6    conda-forge

Environment info

active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /root/.condarc
 populated config files : /opt/conda/.condarc
          conda version : 23.5.0
    conda-build version : not installed
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.31=0
                          __linux=5.10.16.3=0
                          __unix=0=0
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/23.5.0 requests/2.31.0 CPython/3.10.12 Linux/5.10.16.3-microsoft-standard-WSL2 ubuntu/20.04.6 glibc/2.31
                UID:GID : 0:0
             netrc file : None
           offline mode : False
gillins commented 1 year ago

Can you try installing the compilers into a new environment and activate that first?

leonid-butenko commented 1 year ago

Can you try installing the compilers into a new environment and activate that first?

Same result:

$> docker run -it --rm -v /work:/work condaforge/miniforge3
(base) root@87026bdd2af5:/# conda create -n test -y gcc_linux-64  gxx_linux-64 libgdal
...
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate test
#
# To deactivate an active environment, use
#
#     $ conda deactivate
(base) root@87026bdd2af5:/# conda activate test
(test) root@87026bdd2af5:/# cd /work
(test) root@87026bdd2af5:/work# $GXX -I$CONDA_PREFIX/include -L$CONDA_PREFIX/lib -lgdal test_gdal.cpp -o test_gdal

test_gdal.cpp: In constructor 'Interpolator2D::Interpolator2D(const std::vector<double>&, const std::vector<double>&, const std::vector<std::vector<double> >&)':
test_gdal.cpp:39:25: warning: ignoring return value of 'CPLErr GDALRasterBand::RasterIO(GDALRWFlag, int, int, int, int, void*, int, int, GDALDataType, GSpacing, GSpacing, GDALRasterIOExtraArg*)' declared with attribute 'warn_unused_result' [-Wunused-result]
   39 |         poBand->RasterIO(GF_Write, 0, 0, nXSize, nYSize, flatData.data(), nXSize, nYSize, GDT_Float64, 0, 0);
      |         ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_gdal.cpp: In member function 'double Interpolator2D::interpolate(double, double)':
test_gdal.cpp:48:28: warning: ignoring return value of 'int GDALInvGeoTransform(double*, double*)' declared with attribute 'warn_unused_result' [-Wunused-result]
   48 |         GDALInvGeoTransform(adfGeoTransform, inv_geotransform);
      |         ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_gdal.cpp:62:25: warning: ignoring return value of 'CPLErr GDALRasterBand::RasterIO(GDALRWFlag, int, int, int, int, void*, int, int, GDALDataType, GSpacing, GSpacing, GDALRasterIOExtraArg*)' declared with attribute 'warn_unused_result' [-Wunused-result]
   62 |         poBand->RasterIO(GF_Read, x, y, 2, 2, adfPixel, 2, 2, GDT_Float64, 0, 0);
      |         ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/conda/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/13.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: /opt/conda/envs/test/lib/libgdal.so: undefined reference to `posix_spawnp@GLIBC_2.15'
/opt/conda/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/13.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: /opt/conda/envs/test/lib/libgdal.so: undefined reference to `memcpy@GLIBC_2.14'
collect2: error: ld returned 1 exit status
gillins commented 1 year ago

Try:

$GXX $CXXFLAGS -I$CONDA_PREFIX/include -L$CONDA_PREFIX/lib -lgdal test_gdal.cpp -o test_gdal $LDFLAGS

(CXXFLAGS doesn't seem strictly needed, but LDFLAGS definitely does).

leonid-butenko commented 1 year ago

Seems to solve the issue, thanks a lot! The option that actually helped to do that is: -Wl,--allow-shlib-undefined

--allow-shlib-undefined     Allow unresolved references in shared libraries

Any ideas why this magic is needed to link gdal library compiled by conda-forge on X86? For linux-aarch64 for example it works without. I believe the cross-platform consistency would be appreciated!

gillins commented 1 year ago

I think it is to do with what version of libc you have installed, but not really my area.

h-vetinari commented 1 year ago

This can be solved by adding

- sysroot_linux-64 2.17  # [linux64]

to the build environment.

unkaktus commented 10 months ago

Thanks, @h-vetinari ! Indeed, the default sysroot is too outdated to have the symbol.