corteva / rioxarray

geospatial xarray extension powered by rasterio
https://corteva.github.io/rioxarray
Other
504 stars 80 forks source link

Raster files opened with rioxarray are not closed successfully. #701

Open riftia opened 9 months ago

riftia commented 9 months ago

Consider the following workflow:

import rioxarray as rxr

raster_files = glob.glob("/path/to/files") # 50 raster files

for f in raster_files:
    with rxr.open_rasterio(f) as ds:
        # perform some operations and write out a new raster
        ds_max = ds.max(dim='band')
        ds_max.rio.to_raster("/path/to/output/file")

Running this code causes the disk throughput and I/O to rise exponentially after about 20 files have been iterated over, leading to the system freezing. Since the files are being opened within a context manager, they should automatically close.

I tried recreating the same workflow using xarray, and faced no problems.

import xarray as xr

raster_files = glob.glob("/path/to/files") # 50 raster files

for f in raster_files:
    with xr.open_dataset(f) as ds:
        # perform some operations and write out a new raster
        ds_max = ds.max(dim='band')
        ds_max['band_data'].rio.to_raster("/path/to/output/file")

The above code snippet works just fine.

I also tried explicitly calling ds.close() for rioxarray, but the problem still perists.