sync non-rewarded ballots

[countvonzero]

Description

related https://github.com/spacemeshos/go-spacemesh/issues/3987

from Iddo

for non-rewarded ballots, each honest node sorts them according to local timestamps (so 
the ballot that was received most recently will be the last in the sorting, even if it's a ballot 
of an old layer), but also maintain their hash-chain according to lexicographical ordering. 
For the sync, the neighbor will send a batch of the latest non-rewarded ballots (according 
to his local timestamp ordering), and if the hash-chain value is not agreed upon (in particular 
if the entire batch was already known to the other neighbor) then neighbor will send the 
previous batch, and so on (for example, non-rewarded ballots 1 to 100 were receiving during 
the first three months after genesis, non-rewarded ballots 101 to 200 were received during 
the next two months after that, etc.).  

BTW when neighbor Alice send the batch to her neighbor Bob, then Bob computes the 
symmetric difference (with his own batch) and sends the result back to Alice (the same is 
done with layer J when we do the binary search in the contextually valid blocks case), 
I hope that we do this symmetric difference step?

from Tal

The point is that it's not enough to sync blocks, nodes also have to sync ballots. But not all 
ballots are in consensus. If we use a similar strategy to the block sync, and each node creates 
a cumulative hash of all the ballots it sees in each layer, malicious nodes can "insert" ballots 
into old layers arbitrarily, and cause honest nodes to start a resync arbitrarily far into past

The solution we suggested was to separately sync ballots that are in consensus (which we 
know because the IDs whose proposals were used to create a UCB are recorded in order to 
receive rewards) --- we do this using the cumulative hash. For the remaining ballots (which 
should be far fewer), we sync using a different method that sorts by local timestamps rather 
than the layer number.

from Iddo

it's a network saturation DoS attack, if it's an adversary that claims to be a newcomer then 
that's not an attack because the data sent to him can be throttled by the honest node, but an 
adversary that claims that the honest node doesn't have data (especially non-rewarded ballots) 
can connect to many honest nodes and cause them to crawl. Btw this is related to what we 
discussed last week re reward formula for malicious nodes that claim to see a small active set 
and be eligible to create many ballots, because such an adversary will expend spacetime only 
once to create ATX that's eligible for many ballots that will be non-rewarded but mess up the 
sync of honest nodes.

Implementation plan

TBD

[dshulyak]

current design doesn't guarantee that rewarded ballots will be downloaded either. it guarantees that node will download enough weight to cross threshold in the same direction, so for example it can download 77% of ballots, but there will be no distinction between them. it would be wrong-headed to focus on downloading non-rewarded ballots, without rethinking this more broadly

Iddo shared sync design ideas that used hash based on rewarded ballots (instead of consensus/mesh hash) , this will guarantee downloading rewarded ballots, but leave non-rewarded ballots outside of this protocol. only then it makes sense to focus on downloading non-rewarded ballots

[countvonzero]

it was decided during the research meeting on June 26 that we will

• for genesis we should deploy https://github.com/spacemeshos/go-spacemesh/issues/4603
• Iddo will develop proposal for a new sync spec for implementation post genesis.