search

owid / data-api

API for accessing data from our data catalog.
MIT License
16 stars 4 forks source link

API is consuming ~4GB memory, why that much? #12

Open Marigold opened 2 years ago

Marigold commented 2 years ago

Since our analytics server is quite small, this could become an issue since ETL runs on the same server. There's no reason API should be using that much memory. How is that possible?

larsyencken commented 2 years ago

Hmm... must be some kind of memory leak after serving requests.

Marigold commented 2 years ago

It's rather DuckDB keeping data in cache (I've checked the app and there don't seem to be leaks). By default it uses 13gb max memory, so I reduced it down to 2gb.