Revamp bottom-up interaction timing

[maciejwitowski]

From @raulk

We discussed straightening out the bottom-up interactions in the Tue 12 Mar huddle. Key takeaways and inline CTAs, feel free to dispute/comment if you disagree with something.

Checkpoint trigger. Checkpoints can be sent either:

• opportunistically, when there is another more urgent action to convey bottom-up (a message, a config change ack, and in the future a fault/fraud proof).
  • These actions are transmitted through checkpoints as this is what awards them ordering.
  • Note: we experimented in the past with decoupling "bottom up message bundles" from checkpoints, and we lost ordering guarantees, so we won't go back there.
  • in a timed manner, when bottom_up_check_period has passed since the last checkpoint.

Timer reset. Releasing an opportunistic checkpoint resets the timer.

Naming. bottom_up_check_period should be renamed to max_checkpoint_delay, as it represents the maximum amount of time the subnet can go without a checkpoint.

Bottom-up message batching. We should have a dedicated policy for accumulation/batching and release (by generating a checkpoint). Suggesting these policy attributes:

• immediate_release_threshold: how many messages need to accumulate before we immediately release a batch (deals with bursts).
• max_wait: how long the oldest unreleased message needs to have sit in the postbox before being released (deals with latency).

Ack configuration change. We should release acks to configuration changes immediately. These are rather infrequent and can be time-sensitive (e.g. subnet is under active attack from a malicious validator, and we want to ban them).

See also https://github.com/consensus-shipyard/ipc/issues/648#issue-2109230463

Task list

• [ ] Adjust genesis config
  • [ ] Rename bottom_up_check_period to max_checkpoint_delay_blocks
  • [ ] Also rename in ipc-cli, docs, and elsewhere, but retain the previous name as an alias to not break the world; maybe print a warning if the old flag name is used so we can deprecate it in the future
  • [ ] Add msg_immediate_release_threshold config value.
  • [ ] Add msg_max_wait_bocks config value (as blocks)
• [ ] Gateway actor: track last checkpoint release timer
• [ ] Gateway actor: adjust message accumulation/release logic
  • [ ] Reset the checkpoint timer on release (last_checkpoint)
• [ ] Introduce timer logic in gateway

[maciejwitowski]

From @cryptoAtwill

Just to clarify, so when a user submits a bottom up message, it's first stored in the epoch of the last_checkpoint_height + max_checkpoint_delay, then if the batch is full, i.e. immediate_release_threshold is reached, we shift the batch to the current epoch, like how we are doing it now. There is no change to this logic.