search

tarantool / vshard

The new generation of sharding based on virtual buckets
Other
100 stars 31 forks source link

Improve buckets discovery #257

Open mtrempoltsev opened 3 years ago

mtrempoltsev commented 3 years ago

After this fix https://github.com/tarantool/vshard/issues/210 my application cannot use storages for a very long time after starting:

2020-12-01 14:56:12.624 [8516] main/133/vshard.failover._static_router I> All replicas are ok
2020-12-01 14:56:12.666 [8516] main/134/vshard.discovery._static_router I> Updated replicaset(uuid="3ce3bbc6-641f-42cc-abe0-2c973c6ce89e", master=localhost:3301(admin@localhost:3301)) buckets: was 1000, became 2000
2020-12-01 14:56:13.678 [8516] main/134/vshard.discovery._static_router I> Updated replicaset(uuid="3ce3bbc6-641f-42cc-abe0-2c973c6ce89e", master=localhost:3301(admin@localhost:3301)) buckets: was 2000, became 3000

...

2020-12-01 14:56:40.040 [8516] main/134/vshard.discovery._static_router I> Updated replicaset(uuid="3ce3bbc6-641f-42cc-abe0-2c973c6ce89e", master=localhost:3301(admin@localhost:3301)) buckets: was 28000, became 29000
2020-12-01 14:56:41.052 [8516] main/134/vshard.discovery._static_router I> Updated replicaset(uuid="3ce3bbc6-641f-42cc-abe0-2c973c6ce89e", master=localhost:3301(admin@localhost:3301)) buckets: was 29000, became 30000
2020-12-01 14:56:41.052 [8516] main/134/vshard.discovery._static_router I> Discovery enters idle mode, all buckets are known. Discovery works with 10 seconds interval now

I understand that in the case of a large count of buckets the old algorithm led to a load on the router, but it may be worth considering the count of buckets in the new algorithm?

ochaton commented 3 years ago

@mtrempoltsev You may decrease vshard.consts.DISCOVERY_WORK_INTERVAL = 0.001 for initial discovery. And bring it back when router discovers whole cluster