Submitter history sync protocol

[gnulib]

Objective

Honest nodes with DLT stack should be in sync w.r.t. to transaction history for a submitter. If peer nodes exchange any transaction that has a last submitter transaction not known, they should reject the network transaction and instead initiate a submitter history sync.

Acceptance Criteria

As per "A simple double spending resolution protocol" rules described in #30, when a node receives a network transaction from a peer when the "last transaction" from submitter is not known, a submitter sync will be initiated with the peer as following:

• add the transaction ID to ToBeFetchedStack for the peer's connection
• send a SubmitterHistoryRequest message to the peer with "submitter ID" and previous "submitter Seq" (i.e. current seq - 1)
• when receive a SubmitterHistoryRequest message
  • lookup submitter history for specified "submitter ID" and "submitter seq"
  • respond back with SubmitterHistoryResponse message with all the "transaction id" for that submitter ID and sequence requested
• when receive a SubmitterHistoryResponse message
  • iterate through each transaction, if transaction is not known then add to ToBeFetchedStack
  • if there was no new transaction added to ToBeFetchedStack (i.e. all transactions were known), then
    • pop top item from ToBeFetchedQueue and send a SubmitterTxRequest message to peer with ~Submitter ID, submitter seq and~ tx ID
  • else (i.e. there was at-least one unknown transaction)
  • send a new SubmitterHistoryRequest message to the peer with "submitter ID" and previous "submitter Seq" (i.e. msg seq - 1)
• when receive a SubmitterTxRequest message
  • fetch transaction from local DB and respond back with SubmitterTxResponse message
• when receive a SubmitterTxResponse message
  • handle transaction via endorser layer and sharder layer
  • pop top item from ToBeFetchedQueue and send a SubmitterTxRequest message to peer with ~Submitter ID, submitter seq and~ tx ID

[gnulib]

Revised Submitter Sync Protocol

We need to change above submitter sync protocol so that it does not save the unknown transactions into ToBeFetchedStack while walking up the submitter's history. Instead, we want to walk up the history as much as needed without saving anything (so that no "overflow" attack can be done), then once we find a sequence that has all known transactions, we start processing down the history sequence and process all transaction for that sequence before moving to next (breadth first processing). In this process, if there is any double spending situation found, it will create a double spending alert (causing shard to flush and re-sync) and the submitter should be "black-listed" on the node.

InitiateSubmitterSync

• if submitter's last transaction is unknown, then we'll discard current transaction and transition to WalkUpStage next
• else (i.e. submitter's last transaction is known) then continue processing current transaction

WalkUpStage

• send a SubmitterWalkUpRequestMsg to peer with submitter ID and last sequence
• when receive a SubmitterWalkUpRequestMsg from a peer, respond back with SubmitterWalkUpResponseMsg with all known pairs of shard-id/transaction-id for that submitter ID and seq in request
• when receive a SubmitterWalkUpResponseMsg from peer
  • ~if there is a shard-id that is unknown, initiate a shard sync with peer for that shard-id~
    (if local submitter history does not have anything for this shard at a seq, we just continue walking up)
  • ~else~ if local node does not have a transaction for that shard-id, for the given submitter id/seq, then send another SubmitterWalkUpRequestMsg to peer with submitter ID and last sequence
  • else if local node has a transaction for that shard-id but DOES NOT matches in peer's response, then initiate a double spending alert!!!
  • else (i.e. local node has a transaction for a shard-id that matches peer's response) then ignore that pair
  • if all pairs were known, then transition to ProcessDownStage

ProcessDownStage

• send a SubmitterProcessDownRequestMsg to peer with submitter ID and next seq to fetch all known shard/transaction from peer
• when receive a SubmitterProcessDownRequestMsg from a peer, respond back with a SubmitterProcessDownResponseMsg that has submitter id, seq and all transactions for that submitter/seq
• when receive a SubmitterProcessDownResponseMsg from a peer
  • if response does not have any transactions then EndOfSync
  • else process each transaction as following:
  • if transaction's submitter/seq does not match response's submitter/seq -- disconnect peer and abort
  • if there is a shard-id that is unknown, initiate a shard sync with peer for that shard-id
  • else if local node does not have any transaction for transaction's submitter/seq/shard-id, then ~add the transaction id to SubmitterTxQ to process~ process the transaction as a network transaction
  • else if local node has a transaction for that shard-id but DOES NOT matches in peer's response, then initiate a double spending alert!!!
  • else (i.e. local node has a transaction for a shard-id that matches peer's response) then ignore that pair
  • ~emit a PopSubmitterTxQ event and transition to ProcessSubmitterQueue~ send a SubmitterProcessDownRequestMsg with next higher sequence to peer

---

(All of below is not applicable since we are processing transaction right away and not keeping in any queue for later processing)

~ProcessSubmitterQueue~

• when receive a PopSubmitterTxQ event
  • pop the front transaction from queue
  • if there was an item, send SubmitterTxRequestMsg to peer
  • else (i.e. queue empty) then transition to ProcessDownStage to continue walking down and processing
• when receive SubmitterTxRequestMsg, respond back with transaction details in SubmitterTxResponseMsg
• when receive SubmitterTxResponseMsg
  • process transaction
  • emit a PopSubmitterTxQ event to process next item