Holocene-D: op-batcher: Check existing code paths for ordering violations

[sebastianst]

We should check

• normal operation always enqueues blocks inside in order,
• keeps channels in the channel queue in order
• sends transactions in order
  • also frames within blob transactions in order
• reorg case
• re-queueing case when a channel fails to submit in time -> need to preserver order across channels (also future pending)
  • this is where we need to look closely at changing da-type interactions, also with future channels
• re-queueing case when da-type changes
• possibly other things to check that I don't see right now.

Note that we can optimize for simplicity by trading possible double-submission of data. The DP will just drop future batches, and already included batches are ignored.

[geoknee]

From our call today:

The batcher currently requeues block when a channel times out on chain: https://github.com/ethereum-optimism/optimism/blob/b0a4c11489abc7adfa4e801d6b06fb9e496c200f/op-batcher/batcher/channel_manager.go#L98-L109

This means there is a chance for the blocks queue to get out-of-order, if more blocks have been dequeued since the blocks for the channel in question were dequeued. In turn, this could result in channels being sent out of order. With Holocene, that could cause a safe chain halt because channels are not buffered in the derivation pipeline.

Note that situation only arises when channels have sufficient frames to spill over multiple L1 transactions. This is rare but not impossible or ruled out by the implementation.

One proposed solution to this is, instead of requeueing the blocks in the channel (essentially throwing it out and starting again) we can just simply resend the channel. The protocol allows for channel IDs to be reused.

Another more radical proposal might be to remove channel timeouts altogether. Since the derivation pipeline will (post Holocene) drop frames when new first frames arrive, perhaps it is not necessary for the channel timeout to be there to reduce memory consumption / do garbage collection?