apache / kvrocks

Apache Kvrocks is a distributed key value NoSQL database that uses RocksDB as storage engine and is compatible with Redis protocol.
https://kvrocks.apache.org/
Apache License 2.0
3.47k stars 450 forks source link

Metrics for namespace level #755

Open liangyuanpeng opened 2 years ago

liangyuanpeng commented 2 years ago

Search before asking

Motivation

Ref: https://github.com/apache/incubator-kvrocks/discussions/726 We are using the namespace to separate data between users and it works well. But the metrics from the info command stats didn't separate by namespace as well. As a user, we want to know the speed of commands for each namespace.

Solution

No response

Are you willing to submit a PR?

git-hulk commented 2 years ago

@liangyuanpeng Thanks for your feedback, it makes a lot sense.

zevin02 commented 1 year ago

Do you want only the data of stats to be isolated according to the namespace, or all parameters of the command INFO are required?

jihuayu commented 1 year ago

Now kvrocks don't save stat data for each namespace, if we need make it we should save stat for each namespace.

we can change Server#stats to a map that save namespace to stat mapping.

This will increase additional cost above:

  1. Roughly 2KB memory usage per namespace
  2. lock namespace-stat map when we add namespace
  3. One more indirect addressing for access stat
  4. Administrator info command need to foreach all namespace stat and count sum/average.

For 1rd we won't have thousands of namespace, so this memory usage is not large. For 2nd add namespace is low-frequency operation and it is fast. For 3rd if we think indirect addressing expensive, we can cache it in connect.(I don't think this is necessary) For 4rd this is not important.

I think them is acceptable. So do we need do it?

gofort commented 4 months ago

+1 it would be cool to have this use case is simple: we see the amount of keys is growing, but we have no idea in which namespace this happens, so now we need to check each namespace manually which is not convenient at all if you have a lot of namespaces