search

harmony-one / bounties

Bounty program is to help the community take part in the development of the Harmony blockchain. It covers from core feature to validator tooling, from dApp development to DeFi integration.
MIT License
59 stars 23 forks source link

Implement a sharded network connectivity structure for efficient intra/inter shard communication #52

Open rlan35 opened 3 years ago

rlan35 commented 3 years ago

Description

Harmony is a sharded blockchain where there are 4 shards each with hundreds of nodes doing consensus in parallel. The p2p network behind Harmony is based on libp2p. The network structure now is a fully meshed network where all nodes from all shards are randomly connected with each other without the sharding structure considered. When a validator in a shard needs to broadcast a message to other validators in the same shard, the messages actually travels through all the nodes in the network. This is rather inefficient in the network usage.

Ideally, we need to have a network structure that reflect the sharding structure too where there should be more connections between nodes in the same shards and less connections across shards. When messages are meant only for nodes within a shard, the message shouldn't be broadcasted to nodes from other shards. This will greatly reduce the network cost for regular consensus within shards.

Note this is not about message routing but more about network topology on how nodes are connected. This will require changes on both the bootnode and the peer nodes.

Acceptance Criteria

Reward

$10000 equivalent in ONE tokens.

renlulu commented 3 years ago

By saying network structure, do you mean something like A Kademlia DHT? so basically we could probably assign one DHT per shard which is only for broadcasting within the shard.

rlan35 commented 3 years ago

By saying network structure, do you mean something like A Kademlia DHT? so basically we could probably assign one DHT per shard which is only for broadcasting within the shard.

It's not about message routing. It's about network topology where nodes within the same shards are well connected and there is less connection between shards. This way, message broadcasting can be within a shard and doesn't have to travel through all shards.

gitcoinbot commented 3 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

This issue now has a funding of 126673.7272 ONE (10159.23 USD @ $0.08/ONE) attached to it.

renlulu commented 3 years ago

By saying network structure, do you mean something like A Kademlia DHT? so basically we could probably assign one DHT per shard which is only for broadcasting within the shard.

It's not about message routing. It's about network topology where nodes within the same shards are well connected and there is less connection between shards. This way, message broadcasting can be within a shard and doesn't have to travel through all shards.

Okie, thanks for your clarifying, would like to dive a bit deeper on current implementation to see if I could help on this improvement.

LeoHChen commented 3 years ago

noted, the network topology has to be changed with every resharding configuration. I don't see this design fit before the resharding. It has to go with the design and implementation of resharding.

gitcoinbot commented 3 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 4 weeks from now. Please review their action plans below:

1) voanhcung has started work.

Submit A Plan

Learn more on the Gitcoin Issue Details page.

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

The funding of 126673.7272 ONE (14668.18 USD @ $0.12/ONE) attached to this issue has been cancelled by the bounty submitter

rowlanja commented 2 years ago

Is this issue still open? would be very keen to start working on it