import rioxarray as rxr
raster_files = glob.glob("/path/to/files") # 50 raster files
for f in raster_files:
with rxr.open_rasterio(f) as ds:
# perform some operations and write out a new raster
ds_max = ds.max(dim='band')
ds_max.rio.to_raster("/path/to/output/file")
Running this code causes the disk throughput and I/O to rise exponentially after about 20 files have been iterated over, leading to the system freezing. Since the files are being opened within a context manager, they should automatically close.
I tried recreating the same workflow using xarray, and faced no problems.
import xarray as xr
raster_files = glob.glob("/path/to/files") # 50 raster files
for f in raster_files:
with xr.open_dataset(f) as ds:
# perform some operations and write out a new raster
ds_max = ds.max(dim='band')
ds_max['band_data'].rio.to_raster("/path/to/output/file")
The above code snippet works just fine.
I also tried explicitly calling ds.close() for rioxarray, but the problem still perists.
Consider the following workflow:
Running this code causes the disk throughput and I/O to rise exponentially after about 20 files have been iterated over, leading to the system freezing. Since the files are being opened within a context manager, they should automatically close.
I tried recreating the same workflow using xarray, and faced no problems.
The above code snippet works just fine.
I also tried explicitly calling
ds.close()
for rioxarray, but the problem still perists.