High RAM use during stackstac.stack(items, epsg=item_crs)

gjoseph92 / stackstac

Turn a STAC catalog into a dask-based xarray

https://stackstac.readthedocs.io

MIT License

232 stars 49 forks source link

High RAM use during stackstac.stack(items, epsg=item_crs) #227

Closed J6767 closed 8 months ago

J6767 commented 10 months ago

When I have >150 Sentinel-2 products in my pystac query, and then try to run:

stack = stackstac.stack(items, epsg=item_crs)

I am sometimes encountering >70GB of RAM usage. Is this expected behaviour and is there any workaround? I suspect it stems from the conversion between different CRS but, AFAIK, this is required, as I have found some variations in CRS between S2 products. Many thanks.

gjoseph92 commented 10 months ago

This is definitely not expected behavior. Can you provide a reproducer and more info about your environment?

gjoseph92 commented 8 months ago

Without a reproducible example, there's not much we can help with here, so I'm going to close this. @J6767, if you'd like to provide a reproducer, feel free to re-open.