Hi,
I'm trying to use reindexer in k8s environment with 1 master and 2 slaves. And I'm using slaves for ha and performance(for select queries). And I'm struggling with some diffuculties.
Sometimes the slave nodes can't replicate some namespaces. I haven't figure out why this happening yet. But this can be happened anytime in the future. So if I query a namespace on a slave that hasn't replicated that namespace yet, it should redirect the query to a node which has the namespace and ready to query on that namespace. In a replication environments like redis or elasticsearch, you send a query to any node, and it can handle the query or redirect request to a valid node.
I can check everytime if the namespace's hash is same as master's hash, and if it isn't same, I can query on another node. And this hash cheking loop continues to all slave nodes until I found a slave node that has namespace with same hash with master. But this leads to performance issues.
I saw replication statuses on #memstats like "idle", "syncing", etc. I can also check this status everytime on every query, but it is same logic as I described above and it can lead to performance issues. Also this requires an extra sql parser to determine the target namespace in the query.
Hi, I'm trying to use reindexer in k8s environment with 1 master and 2 slaves. And I'm using slaves for ha and performance(for select queries). And I'm struggling with some diffuculties.
Sometimes the slave nodes can't replicate some namespaces. I haven't figure out why this happening yet. But this can be happened anytime in the future. So if I query a namespace on a slave that hasn't replicated that namespace yet, it should redirect the query to a node which has the namespace and ready to query on that namespace. In a replication environments like redis or elasticsearch, you send a query to any node, and it can handle the query or redirect request to a valid node.
I can check everytime if the namespace's hash is same as master's hash, and if it isn't same, I can query on another node. And this hash cheking loop continues to all slave nodes until I found a slave node that has namespace with same hash with master. But this leads to performance issues.
I saw replication statuses on #memstats like "idle", "syncing", etc. I can also check this status everytime on every query, but it is same logic as I described above and it can lead to performance issues. Also this requires an extra sql parser to determine the target namespace in the query.