ICON BTP Message Fragmentaion

cow-hs commented 3 years ago

iip: <to be assigned>
title: ICON BTP - MessageFragmentation
author: heonseung lee (@leeheonseung)
discussions-to: https://github.com/icon-project/IIPs/issues/tbd
status: Draft
type: Standards Track
category IRC
created: 2020-06-14
requires IIP-25

Simple Summary

If the size of the relay message containing the block proof of the SRC chain exceeds the transaction size allowed by the target chain, define and add logic to fragment the BTP message from the BMC to handle this.

Abstract

If the size of Relay Message containing Block Proof larger than Transaction size of target chain, a problem occurs. To solve this, add the logic to fragment and assemble the Relay Message.

Motivation

Block proof of the ICON chain can be verified in the BMV of the chain using only the parts related to BTP. However, in certain chains, it is sometimes necessary to receive and verify the entire Block Proof.

At this time, the ICON Relay Sender fragments the RelayMessage and delivers it to the ICON BMC, and it is necessary to assemble and verify it.

For this reason, there is no issue for ICON Block Proof verification on target chain, so BTP 1.1 implements it only on the ICON BMC.

Specification

Relay

Interface

segment

func (s *sender) Segment(rm *module.RelayMessage, height int64) ([]*module.Segment, error)

Description:
- Fragment Relay Message as much as Transaction Limit size.

relay

func (s *sender) Relay(segment *module.Segment) (module.GetResultParam, error)

Description:
- If the RelayMessage size containing Block Proof exceeds the transaction to be transmitted, the RelayMessage is fragmented and delivered according to the size.
- If not exceeded, it is transmitted as a normal RelayMessage.
Definition of index
- Indicates the sequence of fragment messages.
- The first index is total message count * -1
- The last index is 0

sendFragment

func (s *sender) sendFragment(rmp *BMCRelayMethodParams, idx int) (module.GetResultParam, error)

Description:
- It indexes the fragmented message separated through the segment and transmits it to the targetchain.

BMC

Interface

handleFragment

void handleFragment(String _prev, String _msg, int _idx);

Params
- _prev: String (BTP Address of the previous BMC)
- _msg: String (Fragmented base64 encoded string of serialized bytes of Relay Message)
- _idx: Integer (Index of fragment)
Description:
- Concat the BTP messages that come with _idx in the order of _idx.
- If _idx < 0, it is determined as the first message, and if _idx == 0, it is determined as the last fragmentaion message.
- Eg)
  - -3, 1, 2, 0
  - -1, 0
- Instead of verifying each time each fragmentation message is processed, it is assembled and verified when the last fragmentation message arrives.
- Assemble fragments of the Relay Message and call by BMC::handleRelayMessage.
- It's allowed to be called by registered Relay.

Implementation

TBD

Copyright

Copyright and related rights waived via CC0.

trmaphi commented 3 years ago

As working on BTP implementation, these are some of my thoughts about this IRC:

ICONDAO team currently avoids transaction size problem by limit each BTP message should contain a limited number of blocks. And the minimum is one Block.
handleFragment should only be used, if and only if the BTP message contains only one BlockUpdate with the corresponding BlockProof, ReceiptProofs, with the BTP message size exceeds the transaction size of the Targeted network.
In the case of submitting a list of Fragments for a given block and the BMR scheduling algorithm triggered, the next-in-order BMR should continue to submit the next Fragment of the given block. As the result, BMCStatusLink must let all BMRs notify about
- what the current index of the latest fragment
- what byte size should Fragment created with the formula: BTPMessage size/ Fragment size
Implementation of fragmentation stored should be on BMV. Because, if BMC handle store fragments and concatenate to a full BTPMessage and pass this message to BMV, we still have the problem of the transaction size limit from BMC -> BMV

cow-hs commented 3 years ago

As working on BTP implementation, these are some of my thoughts about this IRC:

ICONDAO team currently avoids transaction size problem by limit each BTP message should contain a limited number of blocks. And the minimum is one Block.

handleFragment should only be used, if and only if the BTP message contains only one BlockUpdate with the corresponding BlockProof, ReceiptProofs, with the BTP message size exceeds the transaction size of the Targeted network.

In the case of submitting a list of Fragments for a given block and the BMR scheduling algorithm triggered, the next-in-order BMR should continue to submit the next Fragment of the given block. As the result, BMCStatusLink must let all BMRs notify about

what the current index of the latest fragment

what byte size should Fragment created with the formula: BTPMessage size/ Fragment size

Implementation of fragmentation stored should be on BMV. Because, if BMC handle store fragments and concatenate to a full BTPMessage and pass this message to BMV, we still have the problem of the transaction size limit from BMC -> BMV

okay I checked. but ICON Network validates transaction size only external call by EOA. all internal call by SCORE doesn't check about transaction size.

dhlee-icon commented 3 years ago

BMR rotate In BMC implementation, when handleFragment() is called, there is no relay rotation, which means BMR doesn't rotate until BMC assembles all the parts of a message. and then it calls handleRelayMessage() with complete message then it rotates BMR.
BMC to BMV message length. In ICON network, the size limit of parameter of intercall of scores is about 10Mib. This fragmentation is considered for Para BMV(on ICON chain, JAVA) so if parachain message length is not over 10Mib at a time, it might not be a problem.

trmaphi commented 2 years ago

@dhlee-icon

In BMC implementation, when handleFragment() is called, there is no relay rotation, which means BMR doesn't rotate until BMC assembles all the parts of a message. and then it calls handleRelayMessage() with complete message then it rotates BMR.

If this BMR decide not to send Fragmentation anymore, Will it will stop other 24 BMRs continue to Relay?

dhlee-icon commented 2 years ago

If this BMR decide not to send Fragmentation anymore, Will it will stop other 24 BMRs continue to Relay?

I don't understand

trmaphi commented 2 years ago

Assume, one BMR 1 needs to send 4 fragments -3, 2, 1, 0. For some technical problem, BMR 1 sent Fragments with indexes -3, 2, then stopped. After some block intervals time, the BMR 2 is supposed to send RelayMessages in the BMR rotation logic. However, due to handleFragment() is called, there is no relay rotation, which means BMR doesn't rotate until BMC assembles all the parts of a message, BMC doesn't allow BMR 2 send RelayMessages anymore.

As this continues, BMR 2 -> 25 won't be allowed to send RelayMessages anymore, as it still BMR 1 turn to RelayMessages with handleFragment() method

dhlee-icon commented 2 years ago

I think in that situation, BMR 1 should either 1) resume calling handleFragment() with part 1 and so on or 2) call handleRelayMessage with new RelayMessage. Anyway, BMC will still look up only BMR1, rejecting any calling from other BMRs including BMR2. But need to check if BMR1 stuck in any situation, BMC can continue to rotate to next BMR

trmaphi commented 2 years ago

I think in that situation, BMR 1 should either 1) resume calling handleFragment() with part 1 and so on or 2) call handleRelayMessage with new RelayMessage. Anyway, BMC will still look up only BMR1, rejecting any calling from other BMRs including BMR2

Doing this makes BMC become vulnerable of being controlled by BMR1. For example: If BMR1 shutdowns for 7 days, the BTP messages can't be transferred for 7 days.

dhlee-icon commented 2 years ago

need to check if BMR1 stuck in any situation, BMC can continue to rotate to next BMR

trmaphi commented 2 years ago

need to check if BMR1 stuck in any situation, BMC can continue to rotate to next BMR

So, BMC will need to make a time limit like for N amount of Block Intervals, if BMR1 can't send Fragment anymore, BMC continue to next BMR?

dhlee-icon commented 2 years ago

here is the conclusion. If RelayMessage contain BTP Message, next BMR will take turn after Delaylimit. If RelayMessage doesn't contain BTP Message and have only BlockUpdates, next BMR will take turn after rotationTerm. As a result, In the case you mentioned above, BMR2 will take turn after some time.

I think in that situation, BMR 1 should either 1) resume calling handleFragment() with part 1 and so on or 2) call handleRelayMessage with new RelayMessage. Anyway, BMC will still look up only BMR1, rejecting any calling from other BMRs including BMR2. But need to check if BMR1 stuck in any situation, BMC can continue to rotate to next BMR

But as what I mentioned above, BMR would better do such recovery action to prevent unnecessary idle time.

quocle108 commented 2 years ago

I think in that situation, BMR 1 should either 1) resume calling handleFragment() with part 1 and so on or 2) call handleRelayMessage with new RelayMessage. Anyway, BMC will still look up only BMR1, rejecting any calling from other BMRs including BMR2. But need to check if BMR1 stuck in any situation, BMC can continue to rotate to next BMR

@dhlee-icon does it allow the next BMR can relay if BMR1 has stuck a long time? If so,i think It's pretty centralized if the system only wating for BMR1 , am i right?

dhlee-icon commented 2 years ago

@quocle108 please refer to answer above

trmaphi commented 2 years ago

Assume, one BMR 1 needs to send 4 fragments -3, 2, 1, 0.

For some technical problem, BMR 1 sent Fragments with indexes -3, 2, then stopped. After some time, the BMR 2 is supposed to send RelayMessages in the BMR rotation logic.

If RelayMessage contain BTP Message, next BMR will take turn after Delaylimit. If RelayMessage doesn't contain BTP Message and have only BlockUpdates, next BMR will take turn after rotationTerm. As a result, In the case you mentioned above, BMR2 will take turn after some time.

I don't understand quite clearly, in this situation BMR2 will submit the Fragmentations -3, 2, 1, 0, right?

dhlee-icon commented 2 years ago

@trmaphi As you know, BMC just take turns to next BMR. It is BMR which is in charge of sending BTP Message. As long as BMR2 send right call, BMC will accept it.Why don't you go on a test with it

trmaphi commented 2 years ago

@dhlee-icon, sure, I'll make a manual test for this case. And come back with the result 👌

icon-project / IIPs