Example on using aiomcache for a cluster

[duxing]

Thanks for building this amazing lib! I was wondering if you can share an example for using aiomcache to connect to a memcached cluster with more than one host?

Seems like this is how connection are established and I don't see any other source code for connection management for multi-node cluster. Given how widely memcached clusters are used, I think an example would be really helpful

[takeda]

@duxing I was looking at the source code, as I'm still wondering if I should use aiomcache, emcache or pymemcache (probably will end up with emcache as pymemcache provides most features, but emcache is async, and aiomcache doesn't appear to have cluster support). I don't see any hashing code. I doubt this library provides clustering support at the time of writing this.

[duxing]

@takeda I did the same assessment and I prefer the features and performance of aiomcache more than other options. I briefly went through the source code and didn't see the client-side of hashing code and I haven't continued to figure out the set up for a multi-node cluster. Not having that support would be a deal breaker for production adoption and I really wish it can be supported

[Dreamsorcerer]

But, if anyone wants to work on it, we can certainly accept new PRs.

[takeda]

The hash computation can be expensive, especially with large number of nodes, because (as I understand from looking at emcache code) the hashing is done for every node, and then node with highest score is picked up. That could be the reason why it is slower.

I do like aiomcache interface, get, and multi_get feels clearer than get and get_multi.

[STOCKZE]

same here, looking for hashing - cluster - hashring implementation through aiomcache

import uhashring
from aiocache import caches, cached, AioCache

# Define your memcached server list
MEMCACHED_SERVERS = [
    {"host": "server1", "port": 11211},
    {"host": "server2", "port": 11211},
    # Add more servers as needed
]

# Create a hashring instance
hash_ring = uhashring.HashRing(initial_servers=MEMCACHED_SERVERS)

# Define a custom cache backend class with hashring
class MemcachedHashRingBackend(AioCache):
    async def get(self, key):
        server = hash_ring.get_node(key)  # Get server using hashring
        async with caches[server["host"]].get(key) as value:
            return value

    async def set(self, key, value, ttl=None):
        server = hash_ring.get_node(key)
        await caches[server["host"]].set(key, value, ttl)

    async def delete(self, key):
        server = hash_ring.get_node(key)
        await caches[server["host"]].delete(key)

# Configure aiocache with the custom backend
cache = caches.get("my_cache", backend=MemcachedHashRingBackend)

# Example usage with caching
@cached(ttl=60, cache=cache)
async def fetch_data(key):
    # Simulate data fetching
    data = f"Data for key {key}"
    return data

async def main():
    data = await fetch_data("my_key")
    print(data)

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())

[STOCKZE]

https://github.com/facebook/mcrouter/issues/368#issuecomment-2218294047 mcrouter has raft / consensus while replicating as seen from this reply