search

conda-forge / rasterio-feedstock

A conda-smithy repository for rasterio.
BSD 3-Clause "New" or "Revised" License
44 stars 23 forks source link

Memory leaks #323

Open sgillies opened 5 days ago

sgillies commented 5 days ago

Solution to issue cannot be found in the documentation.

Issue

Rasterio users are reporting memory leaks with recent conda-forge packages. For example https://github.com/rasterio/rasterio/issues/3250. It doesn't appear to be a problem with Rasterio wheels on PyPI. GeoTIFFs are involved, so the actual problem could be in libtiff or in the new packaging of GDAL.

@akrherz et al, I'll add more details as I find them.

Installed packages

rasterio

Environment info

rasterio info:
  rasterio: 1.4.2
      GDAL: 3.9.3
      PROJ: 9.5.0
      GEOS: 3.13.0
 PROJ DATA: /home/brandon/git/gff/envs/gff2/share/proj
 GDAL DATA: /home/brandon/git/gff/envs/gff2/share/gdal

System:
    python: 3.11.10 | packaged by conda-forge | (main, Oct 16 2024, 01:27:36) [GCC 13.3.0]
executable: /home/brandon/git/gff/envs/gff2/bin/python
   machine: Linux-6.8.0-48-generic-x86_64-with-glibc2.35

Python deps:
    affine: 2.4.0
     attrs: 24.2.0
   certifi: 2024.08.30
     click: 8.1.7
     cligj: 0.7.2
    cython: None
     numpy: 2.1.3
click-plugins: None
setuptools: 75.3.0
AndrewAnnex commented 3 days ago

I also have an interest in seeing this fixed, so I decided to take a look at how the dependencies changed over time and to contrast with the pinned library versions in rasterio-wheels

I am attempting to do a more comprehensive export of the dependency versions for the various libgdal releases, but some thing I noticed very quickly was that the affected rasterio/gdal environments were using libtiff 4.7.0 while the rasterio-wheels are using 4.3.0 as per the env_vars file. If that's not the case let me know.

Hard to know if that's the actual root cause out-of hand but that seems like a good place to look.