search

conda-forge / rasterio-feedstock

A conda-smithy repository for rasterio.
BSD 3-Clause "New" or "Revised" License
42 stars 22 forks source link

Compound CRS segmentation fault #282

Closed dugalh closed 9 months ago

dugalh commented 9 months ago

Solution to issue cannot be found in the documentation.

Issue

Hello. I get a segmentation fault when using a compound CRS with the latest conda-forge rasterio.

import rasterio as rio
crs = rio.CRS.from_string('EPSG:32634+5773')
print(crs)

It works OK with pyproj.

import pyproj
crs = pyproj.CRS.from_string('EPSG:32634+5773')
print(crs)

I didn't have this problem with previous versions of conda-forge rasterio, and don't see it with the latest PyPI rasterio, so am raising it here...

Installed packages

# packages in environment at /home/dugalh/miniconda3/envs/py311-geospatial:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
affine                    2.4.0              pyhd8ed1ab_0    conda-forge
alsa-lib                  1.2.10               hd590300_0    conda-forge
aom                       3.5.0                h27087fc_0    conda-forge
attr                      2.5.1                h166bdaf_1    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
blosc                     1.21.5               h0f2a231_0    conda-forge
boost-cpp                 1.78.0               h2c5509c_4    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.19.1               hd590300_0    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cairo                     1.16.0            h0c91306_1017    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cfitsio                   4.3.0                hbdc6101_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
dav1d                     1.2.1                hd590300_0    conda-forge
dbus                      1.13.6               h5008d03_3    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
ffmpeg                    6.0.0           gpl_h14e97fc_104    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freeglut                  3.2.2                hac7e632_2    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
freexl                    2.0.0                h743c826_0    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
geos                      3.12.0               h59595ed_0    conda-forge
geotiff                   1.7.1               hee599c5_13    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
glib                      2.78.0               hfc55251_0    conda-forge
glib-tools                2.78.0               hfc55251_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
gnutls                    3.7.8                hf3e180e_0    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
gst-plugins-base          1.22.6               h8e1006c_0    conda-forge
gstreamer                 1.22.6               h98fc4e7_0    conda-forge
harfbuzz                  8.2.1                h3d44ed6_0    conda-forge
hdf4                      4.2.15               h501b40f_6    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
jasper                    4.0.0                h32699f2_1    conda-forge
json-c                    0.17                 h7ab15ed_0    conda-forge
kealib                    1.5.1                hcd42e92_5    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lame                      3.100             h166bdaf_1003    conda-forge
lcms2                     2.15                 h7f713cb_2    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20230802.1      cxx17_h59595ed_0    conda-forge
libaec                    1.0.6                hcb278e6_1    conda-forge
libarchive                3.7.2                h039dbb9_0    conda-forge
libass                    0.17.1               h8fe9dca_1    conda-forge
libblas                   3.9.0           18_linux64_openblas    conda-forge
libcap                    2.69                 h0f662aa_0    conda-forge
libcblas                  3.9.0           18_linux64_openblas    conda-forge
libclang                  15.0.7          default_h7634d5b_3    conda-forge
libclang13                15.0.7          default_h9986a30_3    conda-forge
libcups                   2.3.3                h4637d8d_4    conda-forge
libcurl                   8.3.0                hca28451_0    conda-forge
libdeflate                1.19                 hd590300_0    conda-forge
libdrm                    2.4.114              h166bdaf_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.12               hf998b51_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libflac                   1.4.3                h59595ed_0    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgcrypt                 1.10.1               h166bdaf_0    conda-forge
libgdal                   3.7.2                h3aa23ec_3    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libglib                   2.78.0               hebfc3b9_0    conda-forge
libglu                    9.0.0             hac7e632_1003    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libgpg-error              1.47                 h71f35ed_0    conda-forge
libhwloc                  2.9.2           default_h554bfaf_1009    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libidn2                   2.3.4                h166bdaf_0    conda-forge
libjpeg-turbo             2.1.5.1              hd590300_1    conda-forge
libkml                    1.3.0             h37653c0_1015    conda-forge
liblapack                 3.9.0           18_linux64_openblas    conda-forge
liblapacke                3.9.0           18_linux64_openblas    conda-forge
libllvm15                 15.0.7               h5cf9203_3    conda-forge
libnetcdf                 4.9.2           nompi_h80fb2b6_112    conda-forge
libnghttp2                1.52.0               h61bc06f_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libogg                    1.3.4                h7f98852_1    conda-forge
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libopencv                 4.8.0           py311h643827b_4    conda-forge
libopenvino               2023.0.2             h59595ed_0    conda-forge
libopenvino-auto-batch-plugin 2023.0.2             h59595ed_0    conda-forge
libopenvino-auto-plugin   2023.0.2             h59595ed_0    conda-forge
libopenvino-hetero-plugin 2023.0.2             h59595ed_0    conda-forge
libopenvino-intel-cpu-plugin 2023.0.2             h59595ed_0    conda-forge
libopenvino-intel-gpu-plugin 2023.0.2             h59595ed_0    conda-forge
libopenvino-ir-frontend   2023.0.2             h59595ed_0    conda-forge
libopenvino-onnx-frontend 2023.0.2             h59595ed_0    conda-forge
libopenvino-paddle-frontend 2023.0.2             h59595ed_0    conda-forge
libopenvino-pytorch-frontend 2023.0.2             h59595ed_0    conda-forge
libopenvino-tensorflow-frontend 2023.0.2             h59595ed_0    conda-forge
libopenvino-tensorflow-lite-frontend 2023.0.2             h59595ed_0    conda-forge
libopus                   1.3.1                h7f98852_1    conda-forge
libpciaccess              0.17                 h166bdaf_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libpq                     15.4                 hfc447b1_0    conda-forge
libprotobuf               4.23.4               hf27288f_6    conda-forge
librttopo                 1.1.0               hb58d41b_14    conda-forge
libsndfile                1.2.2                hbc2eb40_0    conda-forge
libspatialite             5.1.0                h090f1da_0    conda-forge
libsqlite                 3.43.0               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libsystemd0               254                  h3516f8a_0    conda-forge
libtasn1                  4.19.0               h166bdaf_0    conda-forge
libtiff                   4.6.0                h29866fb_1    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libva                     2.20.0               hd590300_0    conda-forge
libvorbis                 1.3.7                h9c3ff4c_0    conda-forge
libvpx                    1.13.0               hcb278e6_0    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxkbcommon              1.5.0                h5d7e998_3    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzip                    1.10.1               h2629f0a_3    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
minizip                   4.0.1                h0ab5242_4    conda-forge
mpg123                    1.31.3               hcb278e6_0    conda-forge
mysql-common              8.0.33               hf1915f5_4    conda-forge
mysql-libs                8.0.33               hca2cd23_4    conda-forge
ncurses                   6.4                  hcb278e6_0    conda-forge
nettle                    3.8.1                hc379101_1    conda-forge
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.92                 h1d7d5a4_0    conda-forge
numpy                     1.26.0          py311h64a7726_0    conda-forge
ocl-icd                   2.3.1                h7f98852_0    conda-forge
ocl-icd-system            1.0.0                         1    conda-forge
opencv                    4.8.0           py311h38be061_4    conda-forge
openh264                  2.3.1                hcb278e6_2    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.3                hd590300_0    conda-forge
p11-kit                   0.24.1               hc5aa10d_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
poppler                   23.08.0              ha962622_1    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
postgresql                15.4                 h8972f4a_0    conda-forge
proj                      9.3.0                h1d62c97_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pugixml                   1.13                 h59595ed_1    conda-forge
pulseaudio-client         16.1                 hb77b528_5    conda-forge
py-opencv                 4.8.0           py311h781c19f_4    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pytest                    7.4.2              pyhd8ed1ab_0    conda-forge
python                    3.11.5          hab00c5b_0_cpython    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pyyaml                    6.0.1           py311h459d7ec_1    conda-forge
qt-main                   5.15.8              hc47bfe8_16    conda-forge
rasterio                  1.3.8           py311h40fbdff_2    conda-forge
readline                  8.2                  h8228510_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
snuggs                    1.4.7                      py_0    conda-forge
sqlite                    3.43.0               h2c6b66d_0    conda-forge
svt-av1                   1.7.0                h59595ed_0    conda-forge
tbb                       2021.10.0            h00ab1b0_0    conda-forge
tiledb                    2.16.3               hf0b6e87_3    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
tzcode                    2023c                h0b41bf4_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
x264                      1!164.3095           h166bdaf_2    conda-forge
x265                      3.5                  h924138e_3    conda-forge
xcb-util                  0.4.0                hd590300_1    conda-forge
xcb-util-image            0.4.0                h8ee46fc_1    conda-forge
xcb-util-keysyms          0.4.0                h8ee46fc_1    conda-forge
xcb-util-renderutil       0.3.9                hd590300_1    conda-forge
xcb-util-wm               0.4.1                h8ee46fc_1    conda-forge
xerces-c                  3.2.4                hac6953d_3    conda-forge
xkeyboard-config          2.39                 hd590300_0    conda-forge
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-inputproto           2.3.2             h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.6                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxi                1.7.10               h7f98852_0    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xf86vidmodeproto     2.3.1             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

Environment info

active environment : py311-geospatial
    active env location : /home/dugalh/miniconda3/envs/py311-geospatial
            shell level : 2
       user config file : /home/dugalh/.condarc
 populated config files : /home/dugalh/.condarc
          conda version : 23.3.1
    conda-build version : not installed
         python version : 3.9.16.final.0
       virtual packages : __archspec=1=x86_64
                          __cuda=11.6=0
                          __glibc=2.35=0
                          __linux=5.15.90.1=0
                          __unix=0=0
       base environment : /home/dugalh/miniconda3  (writable)
      conda av data dir : /home/dugalh/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/dugalh/miniconda3/pkgs
                          /home/dugalh/.conda/pkgs
       envs directories : /home/dugalh/miniconda3/envs
                          /home/dugalh/.conda/envs
               platform : linux-64
             user-agent : conda/23.3.1 requests/2.28.1 CPython/3.9.16 Linux/5.15.90.1-microsoft-standard-WSL2 ubuntu/22.04.3 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
akrherz commented 9 months ago

Thank you for reporting this, sadly, I am not immediately reproducing, but don't doubt there is some issue here.

$ python
Python 3.11.5 | packaged by conda-forge | (main, Aug 27 2023, 03:34:09) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rasterio as rio
>>> crs = rio.CRS.from_string('EPSG:32634+5773')
>>>

highlights of my env

gdal                      3.7.2           py311h815a124_3    conda-forge
rasterio                  1.3.8           py311h40fbdff_2    conda-forge
proj                      9.3.0                h1d62c97_1    conda-forge

Do you get any dmesg / syslog output or any other messages when the segfault occurs?

akrherz commented 9 months ago

@dugalh , do you get a segfault with a simple and new environment? I don't :(

$ mamba create -n dev python=3.11 rasterio
$ conda activate dev
$ python
Python 3.11.5 | packaged by conda-forge | (main, Aug 27 2023, 03:34:09) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rasterio as rio
>>> crs = rio.CRS.from_string('EPSG:32634+5773')
>>>
dugalh commented 9 months ago

@akrherz Thanks for the quick reply. It only segfaults when the CRS is used e.g. on print(crs). I do see it in a new rasterio only environment.

This seems to be the relevant part of the dmesg output:

kern  :info  : [Sep23 13:38] python[48228]: segfault at 0 ip 00007ff6a82a49bd sp 00007ffcc3818cb8 error 4 in libc.so.6[7ff6a81>
kern  :info  : [  +0.000030] Code: f8 77 c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 89 f8 48 89 fa c5 f9 ef c0 25 ff 0f 00 0>
kern  :info  : [  +0.000007] potentially unexpected fatal signal 11.
kern  :warn  : [  +0.000001] CPU: 4 PID: 48228 Comm: python Not tainted 5.15.90.1-microsoft-standard-WSL2 #1
kern  :warn  : [  +0.000017] RIP: 0033:0x7ff6a82a49bd
kern  :warn  : [  +0.000004] Code: f8 77 c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 89 f8 48 89 fa c5 f9 ef c0 25 ff 0f 00 0>
kern  :warn  : [  +0.000001] RSP: 002b:00007ffcc3818cb8 EFLAGS: 00010283
kern  :warn  : [  +0.000002] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00005602029b2918
kern  :warn  : [  +0.000001] RDX: 0000000000000000 RSI: 00007ff660861770 RDI: 0000000000000000
kern  :warn  : [  +0.000047] RBP: 0000000000000064 R08: 00007ff6a7e5c000 R09: f4dca66451297fe6
kern  :warn  : [  +0.000002] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
kern  :warn  : [  +0.000001] R13: 00005602037c6010 R14: 00007ff69ef19b40 R15: 00007ff660861780
kern  :warn  : [  +0.000001] FS:  00007ff6a80fd740 GS:  0000000000000000
akrherz commented 9 months ago

It only segfaults when the CRS is used e.g. on print(crs)

I'm sorry, I totally missed that and indeed, that reproduces :(

akrherz commented 9 months ago

I'm sure #275 is at blame and will backport a patch with rasterio/rasterio#2875 included now.

dugalh commented 9 months ago

Thank you...

akrherz commented 9 months ago

Thank you...

Thank you for reporting, this was totally my fault. Hopefully will get a green build shortly and get an updated build onto conda-forge later today.

akrherz commented 9 months ago

The moment of truth :)

$ mamba install rasterio=1.3.8=py311h40fbdff_3
$ python
Python 3.11.5 | packaged by conda-forge | (main, Aug 27 2023, 03:34:09) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rasterio as rio
>>> crs = rio.CRS.from_string('EPSG:32634+5773')
>>> print(crs)
COMPD_CS["WGS 84 / UTM zone 34N + EGM96 height",PROJCS["WGS 84 / UTM zone 34N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",21],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","32634"]],VERT_CS["EGM96 height",VERT_DATUM["EGM96 geoid",2005,AUTHORITY["EPSG","5171"]],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Gravity-related height",UP],AUTHORITY["EPSG","5773"]]]
>>>
dugalh commented 9 months ago

Great - thanks :)