apache / incubator-uniffle

Uniffle is a high performance, general purpose Remote Shuffle Service.
https://uniffle.apache.org/
Apache License 2.0
382 stars 149 forks source link

[Improvement] Extend event size threshold metric configuration to any number of levels #498

Open zuston opened 1 year ago

zuston commented 1 year ago

Code of Conduct

Search before asking

What would you like to be improved?

In current codebase, it has three levels metrics of rss.server.event.size.threshold.l1/l2/l3, but this is not enough for production env.

We hope to extend event size threshold metric configuration to any number of levels, like introducing a new config option: rss.server.event.size.threshold.level.ranges=20k,1m,10m,100m,1g,10g, which could cover the multiple levels for better configuring the writing threads number or single buffer flush size.

How should we improve?

No response

Are you willing to submit PR?

advancedxy commented 1 year ago

rss.server.event.size.threshold.level.ranges=20k,1m,10m,100m,1g,10g

So, you want to generate metric name dynamically, something like rss.server.event.size.threshold.{20k,1g}? I agree the fixed three level is limited and we should extend it. The problem is how much flexibility exposed to end user without confusing them.

zuston commented 1 year ago

So, you want to generate metric name dynamically, something like rss.server.event.size.threshold.{20k,1g}?

Not accurate. Metrics will be exposed like: event_size_range_0_20k, event_size_range_20k_1m ....