spacemeshos / go-spacemesh

Go Implementation of the Spacemesh protocol full node. 💾⏰💪
https://spacemesh.io
MIT License
748 stars 211 forks source link

use fixed precision math in consensus code #4268

Open countvonzero opened 1 year ago

countvonzero commented 1 year ago

Description

from Iddo

We will need yellowpaper that describes our precise protocol with low-level details, 
and we shouldn't write in yellowpaper that the reward calculation is according to 
golang bigint and expect other people to examine how golang implemented it to be 
able to write a compatible algorithm (what could easily happen is that an alternative 
implementation of our protocol will use bigint of another programming language 
which has no guarantee to be compatible). Also golang might upgrade their bigint 
library and introduce small changes that will cause conensus faults for us. 

from Tal

The point we're trying to make is that when you're designing the spec, it's important to
take into account the simplicity of the implementations and how easy it is to verify them. 
By only using 64-bit precision in the spec, we can ensure that implementations can be 
simple and easy to verify, since they won't require additional libraries. Of course, you 
could still choose to implement them using arbitrary precision, but then it really is an 
implementation detail, and not a protocol design choice.

Here's an example of such a spec:

1. Set scale to be the smallest integer so that both total_reward >> scale and user_weight >> scale fit in a u32.
2. Compute user_reward=floor((total_reward >> scale) * (user_weight >> scale) / total_weight)<<(scale*2)
3. Compute leftover rewards (r = total_reward - sum(user_reward[0..n]), r is given to user X)

This spec ensures that the result of the multiplication is always less than 2^64, so it can be 
implemented directly using fixed-precision integer operations (in hardware on 64-bit platforms).
dshulyak commented 1 year ago

i think this excerpt is not sufficient to say that we specified it :). beside this part there is also other consensus code that might not be doing the best thing.

rewards

this is a multistep process. one step is to encode ballot weight into block. second step is to distribute issuance according to those weights. in both steps we are using big.Rat.

beacon

beacon uses big.Float and big.Rat for some computations.

vesting/issuance

vesting uses big.Int. i probably just need to check if we can compute same thing using u64 without overflow. issuance is independent from implementation, we could create schedule using matlab or anything.

tortoise

tortoise uses fixed72/56 for counting weight, including thresholds. would be nice to switch to u64

dshulyak commented 1 year ago

rewards

scale=7 bits is enough to fit issuance per layer. need to double check this. the question is how do we want to encode user_weight. right now it is represented as rational number (with u64 for numerator and denominator). computed as atx_weight ballot_eligibilities / total_eligibilities. do we want too keep only one u64, so computation will be floor(atx_weight ballot_eligibilities / total_eligibilities). it definitely will not overflow, but we will loose some precision

lrettig commented 1 year ago

Regarding issuance per layer, see https://github.com/spacemeshos/economics/blob/main/tenyears.txt. The highest issuance is in the earliest layer, and it's around 477 SMH or 477e9 smidge per layer. This would unfortunately require 39 bits to fully represent.

In [6]: math.log(477e9,2) Out[6]: 38.795198309991775