Running Dragonfly with Memcached in Cluster Mode

[sandeep540]

Did you search GitHub Issues and GitHub Discussions First? Yes

Is your feature request related to a problem? Please describe. Today, if I have to run Memcached as a 3-Node (or N node) Cluster, I need to use external module McRouter (https://github.com/facebook/mcrouter) to run the same, if Dragon fly can replace Memcached and McRouter and provide similar functionality natively, it can replace many memcached instances

Describe the solution you'd like Similar to Redis Cluster, provide memecached Cluster support - https://www.dragonflydb.io/docs/managing-dragonfly/clustermode

Describe alternatives you've considered Currently in K8S this is the solution we consider - https://blog.palark.com/highly-available-memcached-with-mcrouter-in-kubernetes/

Additional context

[romange]

Can you provide more details what exactly needs to be supported? i.e. is there a command, an API that is missing?

[sandeep540]

Currently I cannot run memcached as cluster (with sync), I can only run in single vm/container https://www.dragonflydb.io/docs/managing-dragonfly/running-memcached

Adding cluster feature will be helpful

[chakaz]

Please correct me if I'm wrong, but to my understanding memcached doesn't (directly) support cluster mode as well, right? That's the goal of mcrouter - to route requests to the correct memcached node. Since Dragonfly is API compatible with memcached, you should be able to set up mcrouter that use Dragonfly nodes instead of memcached nodes. Did you try that and experienced some issues?

[sandeep540]

Let me try that in Docker Compose and share my findings, will take this example - https://github.com/StudioSol/docker-mcrouter/blob/master/examples/replication/docker-compose.yml

Not sure how easy/difficult it is? is it possible to bake McRouter functionality into Dragonfly natively! So that dragonfly (with memcached) can be run as cluster

![Uploading Screenshot 2023-07-10 at 2.53.28 PM.png…]()

[sandeep540]

[sandeep540]

I tested with below docker compose, it worked fine https://gist.github.com/sandeep540/ad1a31d6d2187b09ef7bc92a0196fea7

Reasons for removing dependency on McRouter

1. It's heavy, container size is 600MB
2. No ARM Support
3. No active development from Facebook

I like Memcached due to multi-threading, simplicity and speed, if you can make it work, will be all more reason to substitute memcached cluster with Dragonfly

[chakaz]

Unfortunately, we currently do not plan to support extending Dragonfly's functionality to also provide a McRouter-like service.

[sandeep540]

No worries, but can it be one of the backlog items which can be implemented in future

[romange]

Why do you use mcrouter and not connect memcache node directly? I am asking to understand what would be different if we hypothetically bake this functionality into the cluster.

Architecturally speaking you usually want your load-balancer (i.e. mcrouter) be a separate entity from your stateful workloads to allow scaling both of them independently.

[sandeep540]

Memcached servers are not really clustered and the term Memcached cluster, often used by developers is somewhat misleading. Memcached servers do not communicate with each other and in fact, a Memcached server is completely blind to which objects are stored on it, not to mention other servers.

McRouter does 2 things, it lets all the nodes in cluster sync among them, giving true HA and also provide Load Balancer ability. If Dragonfly can replicate true Memcached Cluster (With Sync), Having both Memcached+McRouter it will be worthwhile feature to consider.

I worked in a very big Company in USA (Cannot mention name) where they combined Memcached + McRouter and were giving 3 VM Cluster for cache (it's a heavily customised solution), it was very good with performance and multi threading that they were using it instead of Redis for many critical P0 Applications

DragonFly entire purpose is to replace either Redis (or) Memcached with superior features and performance, 2 leading caching products, this feature which I mentioned might help you get there, hope I could explain with valid points :)