search

tarantool / vshard

The new generation of sharding based on virtual buckets
Other
100 stars 30 forks source link

Create a subscription fiber for each replicaset? #284

Open Gerold103 opened 3 years ago

Gerold103 commented 3 years ago

Currently on the router there is a bucket discovery process which walks all the replicasets and downloads the buckets in a map-reduce-like way.

But it works like polling. It can't subscribe to the changes on the storages to get them immediately. Or at least learn about them immediately. Due to limitations of box.session.push().

The same issue is going to appear with the master discovery process in #75.

If each replicaset would have its own fiber, it could subscribe on changes on the master node of each storage in a blocking netbox call. When there is no master, the fiber could poll the replicas looking for a master. But replicas of only its own replicaset. Not of all of them.

I am not sure though that this is the right way. It needs more thinking.

sergos commented 2 years ago

Can be addressed as part of pub/sub re-work, #313?