Write forum post about why we run out of RAM when doing benchmark queries

azavea / noaa-hydro-data

NOAA Phase 2 Hydrological Data Processing

11 stars 3 forks source link

Write forum post about why we run out of RAM when doing benchmark queries #82

Closed lewfish closed 2 years ago

lewfish commented 2 years ago

We were only able to run the benchmark_queries.ipynbnotebook after increasing worker memory to 4GB (from 2GB), but this is surprising since the chunk sizes were around 100 mb. We should read the documentation more closely to see why this might be the case, and/or write a forum post asking about this.

elizawallace commented 2 years ago

Closing for now pending review of the context for this issue. May be a dask issue.