envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.96k stars 4.8k forks source link

hotkey for redis: find hotkeys by heat #16096

Closed littlegreyzhang closed 3 years ago

littlegreyzhang commented 3 years ago

hotkey for redis: find hotkeys by heat

Can I submit a PR to add the following feature? @mattklein123 @yiyouguisu

Description:

Count some(1\~255) keys of commands. Then convert the frequence to heat(0\~255) by logarithm function.

We can do this in client/proxy/redis. Relative Merits:

client: It's simple and direct. But it need to hack into the client code. And it's difficult to adapt to multi language framework.

proxy: It's completely transparent to client & redis. It can solve the problem of language heterogeneity.

redis(--hotkeys): It will scan the entire keyspace. Therefore, the scanning time is positively correlated with the number of keys.

redis(MONITOR): In some particular cases, running a single MONITOR client can reduce the throughput by more than 50%. Running more MONITOR clients will reduce throughput even more.

We just want to find some hotkeys. Not the heat of all the keys. So I want to use some cache algorithm(such as LFU) to implement this feature in proxy(envoy).

Relevant Links:

https://github.com/redis/redis/pull/4392 https://redis.io/topics/lru-cache https://redis.io/commands/monitor

asraa commented 3 years ago

@msukalski @henryyyang

mattklein123 commented 3 years ago

This sounds like a reasonable feature to add to me.

littlegreyzhang commented 3 years ago

I will do it.

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

github-actions[bot] commented 3 years ago

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.