dao-xyz / peerbit

P2P database framework with encryption, sharding and search
https://peerbit.org
Apache License 2.0
191 stars 15 forks source link

Sharding capacity/resource specific #18

Open marcus-pousette opened 1 year ago

marcus-pousette commented 1 year ago

Right now sharding assume that all peers have more or less same capacity for storage (ram, disc, cpu etc). This can be optimized so that we can use powerful peers more than weak ones.

marcus-pousette commented 1 year ago

Idea:

f ( resources ) -> R^1 

Scale slots with f, so that in the leader election routing, it is more probable to become a leader if you have more resources.

Resource data is shared "when changed" or "when peers connect"

marcus-pousette commented 1 year ago

Network would reach appx census about f, i.e. there would be a deterministic way computational resources ( vector of numbers ) can be translated result in slot width (a number), so that peers will little capacity would be given less responsibility, and more -> more

marcus-pousette commented 1 year ago

i.e.

f: R^n -> R^1

where n is the amount of capacity features.

In the future we can make "f" adaptive to the task at hand. "f" for a high compute, low storage, program would not be the same as for a high storage , low compute, program