netcdf writing finishes faster with a lot of driver memory

Open-EO / openeo-geopyspark-driver

OpenEO driver for GeoPySpark (Geotrellis)

Apache License 2.0

26 stars 4 forks source link

netcdf writing finishes faster with a lot of driver memory #637

Open jdries opened 8 months ago

jdries commented 8 months ago

I observed a batch job running slowly with 5G driver memory setting and finally crashing. Giving it 10G suddenly made it run 3 times faster, this is interesting to incorporate into our default.