apache / brpc

brpc is an Industrial-grade RPC framework using C++ Language, which is often used in high performance system such as Search, Storage, Machine learning, Advertisement, Recommendation etc. "brpc" means "better RPC".
https://brpc.apache.org
Apache License 2.0
16.41k stars 3.96k forks source link

mbvar 如何通过 LatencyRecorder 获取平均耗时 #2660

Open renzhong opened 4 months ago

renzhong commented 4 months ago

Describe the bug (描述bug) mbvar 的 prometheus 的上报格式如下: service_latency{valid="false"} 0 service_latency{valid="false",quantile="80"} 0 service_latency{valid="false",quantile="90"} 0 service_latency{valid="false",quantile="99"} 0 service_latency{valid="false",quantile="999"} 0 service_latency{valid="false",quantile="9999"} 0 service_max_latency{valid="false"} 0 service_qps{valid="false"} 0 service_count{valid="false"} 1 此时没办法通过:avg(service_latency{valid="false"}) 来获取平均耗时,因为 avg 会把所有 service_latency 求平均,包括了 p99等数值。 而 bvar 的 LatencyRecorder 通过拆分名字 latency/latency_99 避免了平均耗时的求 avg 的问题。这应该是 mbvar 的 bug 吧,没办法统计平均耗时了

To Reproduce (复现方法)

Expected behavior (期望行为)

Versions (各种版本) OS: Compiler: brpc: protobuf:

Additional context/screenshots (更多上下文/截图)

renzhong commented 3 months ago

我准备修复这个问题,有两个方案,一个是参考 bvar,拆分名字。一个是把平均耗时改成 service_latency{valid="false",quantile='avg'} 0 这两种方案,哪种更好一些? @wwbmmm

wwbmmm commented 3 months ago

百度内部好像是用的 service_latency{valid="false",quantile="avg"}