mit-dci / opencbdc-tx

A transaction processor for a hypothetical, general-purpose, central bank digital currency
Other
896 stars 198 forks source link

Fix shard-seeder for atomizer #152

Closed wadagso-gertjaap closed 2 years ago

wadagso-gertjaap commented 2 years ago

When using an atomizer configuration with a shard replication factor of >1, the shard seeder will not function properly.

The reason is that the seeder uses the config value m_shard_ranges.size() as the number of logical shards, but in the atomizer situation this vector will contain duplicates.

For 2PC this works, since the shard range is defined on the logical cluster level, so m_shard_ranges contains an element count equal to the logical shard count:

shard_count=4
shard0_count=3
shard0_start=0
shard0_end=63
shard0_0_endpoint="10.0.84.165:5002"
[...]
shard0_1_endpoint="10.10.40.96:5002"
[...]
shard0_2_endpoint="10.20.46.224:5002"
[...]
shard1_count=3
shard1_start=64
shard1_end=127

For Atomizer the range value is defined on individual shards, and are duplicated, so m_shard_ranges contains an element count equal to the physical shard count:

shard0_start=0
shard0_end=63
shard0_endpoint="10.0.84.165:5002"
shard1_start=64
shard1_end=127
shard1_endpoint="10.0.84.111:5002"
[...]
shard4_start=0
shard4_end=63
shard4_endpoint="10.0.84.123:5002"

This PR sorts and de-duplicates the vector of shard ranges to get to the correct count.

HalosGhost commented 2 years ago

Particularly in parts of the code I'm less familiar with, I'd love to have a second set of eyes, so I'll hold off on merging till @metalicjames has a chance to review.

HalosGhost commented 2 years ago

Tested ACK. I've now spent a fair bit more time with the shard-seeder, and this small change (to correctly handle shard replication) has been needed. Looks good to me.