twitter / finagle

A fault tolerant, protocol-agnostic RPC system
https://twitter.github.io/finagle
Apache License 2.0
8.78k stars 1.45k forks source link

Growth of the number of prepared statements on server #935

Open notxcain opened 2 years ago

notxcain commented 2 years ago

Hey folks!

I don't know why but we noticed that PrepareCache in finagle-mysql does not send enough CloseRequests to match PrepareRequests when the cache's capacity gets reached. That leads to a growing number of prepared statements on server. We noticed that it lags behind at around 2-4 rps and this is the exact growth rate we see on the server.

However we see that only in production and I can't reproduce that locally.

Any ideas on why this could it happen?

Don't pay attention to the spikes, it's application restart image

Environment MySQL 8.0 Fiangle 21.8.0