search

envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
25.1k stars 4.82k forks source link

Change Redis cluster discovery to use CLUSTER_SHARDS API #36975

Open milton0825 opened 3 weeks ago

milton0825 commented 3 weeks ago

Title: Change Redis cluster discovery to use CLUSTER_SHARDS API

Description: Currently Envoy uses CLUSTER_SLOTS to fetch cluster topology from Redis. CLUSTER_SLOTS is deprecated in Redis 7.0. It is slower and less efficient than the new API, CLUSTER_SHARDS. We observed CLUSTER_SHARDS payload size to be 10x smaller than the CLUSTER_SLOTS payload for big redis cluster with 12k shards.

Redis doc also recommends using CLUSTER_SHARDS over CLUSTER_SLOTS

[optional Relevant Links:]

Any extra documentation required to understand the issue.

KBaichoo commented 3 weeks ago

cc @mattklein123 @weisisea @msukalski

dceravigupta commented 2 weeks ago

12k shards seems to be a very rare case. Do you happen to know the size difference let say for a 10 shards cache?

weisisea commented 1 week ago

This should involve sending CLUSTER SHARDS commands for topology discovery and parsing the responses similar to how CLUSTER SLOTS responses are parsed.

@KBaichoo could you please help or guide on how to add the "help wanted" label? Thanks!