paypal / junodb

JunoDB is PayPal's home-grown secure, consistent and highly available key-value store providing low, single digit millisecond, latency at any scale.
Apache License 2.0
2.56k stars 163 forks source link

refactor: Improve Zone Initialization and Node Shards Assignment #154

Open KhanSufiyanMirza opened 11 months ago

KhanSufiyanMirza commented 11 months ago

Hi @jostanislas And JunoDb Team this PR Refactor EtcdReader.readNodesShards Method for More Efficient Zone Initialization and Node shards assignment

Description:

Issue Overview:

In the etcd package and file etcdreader.go, the readNodesShards method is responsible for reading Nodes Shards assignment through the etcd reader. During this process, it calls cluster.NewZoneFromConfig to initialize zones if c.Zones[zoneid] == nil. However, there is an opportunity to improve efficiency and avoid redundancy in zone and its nodes initialization. cluster.NewZoneFromConfig not just initialize zone it also execute zone.initShardsAsssignment(numZones, numShards) to populate node but in readNodesShards we override this logic of populating node though this statement c.Zones[zoneid].Nodes[nodeid].StringToNode(uint32(zoneid), uint32(nodeid), string(ev.Value), TagPrimSecondaryDelimiter, TagShardDelimiter)

Proposed Solution:

To enhance code efficiency and eliminate redundancy, I propose refactoring the readNodesShards function. Specifically, I suggest replacing the usage of cluster.NewZoneFromConfig with the new NewZone function. This new function will initialize zones without populating the Nodes field, leaving that task to be performed later in the code when it is needed. By making this change, we can avoid overwriting node data during zone initialization and improve overall efficiency.

Impact:

This change will optimize the readNodesShards function and reduce redundant population of the Nodes field during zone initialization. It is expected to have a positive impact on performance.

Additional Context: This change is suggested as part of ongoing efforts to optimize and improve the codebase of the Juno project. It aims to make zone initialization and Nodes Shards assignment more efficient while maintaining code correctness.

And it Fixes #153