search

k3s-io / kine

Run Kubernetes on MySQL, Postgres, sqlite, dqlite, not etcd.
Apache License 2.0
1.65k stars 237 forks source link

postgres db backend connection pooling #63

Open poblin-orange opened 3 years ago

poblin-orange commented 3 years ago

Hello, using k3s (single server + 2 agents) with an external postgresdb Im encountering an (unrelated) issue with very frequent dns lookup to our dns servers.

Zooming in, i see kine is opening /closing many connections. Is there a way to configure datastore to pool postgresql connections ?

brandond commented 3 years ago

Kine has the capability to tune the connection pooling parameters, but we have not yet exposed these options as k3s command-line flags.

poblin-orange commented 3 years ago

Thks for your answer. Seems the pgx library has this ability. https://github.com/JackC/pgx Any way we could pass this as datastore url and avoid complexifying the k3s command-line ?

brandond commented 3 years ago

No, we're not using that library. We're using github.com/lib/pq.

poblin-orange commented 3 years ago

Ok. note sure how related it is, but switching to mysql gave us a huge perf improvment.

brandond commented 3 years ago

I took a shot at improving compaction performance with the most recent round of updates but I don't think we've baseline benchmarked database performance. Can I ask for more info on the database platforms you switched from and to?

poblin-orange commented 3 years ago

on premise vms databases, provisionned with bosh release:

Single vms db, 2vcpu, 8Go memory. Single vm k3s server, 2 k3s agent vms No real load on k3s server (just rancher and fleet)

brandond commented 3 years ago

Are you just looking at database connections and connection rate to gauge performance? Are you evaluating any of the apiserver datastore performance metrics that it exposes for collection by Prometheus?

poblin-orange commented 3 years ago

no détail monitoring, just saw cpu load improvement