search

canonical / dqlite

Embeddable, replicated and fault-tolerant SQL engine.
https://dqlite.io
Other
3.83k stars 216 forks source link

Investigate growing memory usage found by microk8s benchmarking #494

Open cole-miller opened 1 year ago

cole-miller commented 1 year ago

@ktsakalozos kindly ran some benchmarks of microk8s with different backends, and one of the findings was that the dqlite backend showed steadily increasing memory usage over time, while memory usage for the SQLite and etcd backends was bounded. We should figure out where dqlite is leaking memory -- this shouldn't be so hard to do since we have a reproducible setup: https://github.com/ktsakalozos/k8s-dqlite-stress

ktsakalozos commented 1 year ago

Thank you for opening this issue @cole-miller. I welcome any insight you can give me, but I would not call it a memory leak yet. I think this might be the normal operation where memory usage grows as more data are stored in the DB.

ktsakalozos commented 1 year ago

Many thanks to @berkayoz for coming up with the attached script to stress the memory utilization in a way that does not include any MicroK8s code. mem-reproducer.tar.gz