TipManager: how to deal and (possibly) include invalid blocks (aka. very dirty Tangle)

[piotrm50]

Let's think about this again after the recent changes to the tipmanager.

[jonastheis]

I think we're currently not including invalid blocks or invalid transactions to the TipManager as these are discarded when the invalidity is established, i.e. before reaching the TipManager. The TipManager as such does not need to worry about invalid blocks.

Invalid blocks

• subjective invalidity?
• Blocks are marked as invalid if their parent becomes invalid: https://github.com/iotaledger/iota-core/blob/f94575778de5d9520978f8e038a19ed95ae92578/pkg/protocol/engine/blockdag/inmemoryblockdag/blockdag.go#L96
• Blocks are marked as invalid if they attach to a parent that is too old: https://github.com/iotaledger/iota-core/blob/f94575778de5d9520978f8e038a19ed95ae92578/pkg/protocol/engine/blockdag/inmemoryblockdag/blockdag.go#L227
• Blocks are marked as invalid if their transaction is orphaned and the orphaned slot is below the block's slotcommitment slot: https://github.com/iotaledger/iota-core/blob/7b57e9cf308dc78b15ec9b522f7b0f32e7503dea/pkg/protocol/engine/booker/inmemorybooker/booker.go#L106
• Blocks are marked as invalid if something during booking goes wrong (parent could not be found, containing tx metadata not found, like reference to conflicting tx to containing tx): https://github.com/iotaledger/iota-core/blob/7b57e9cf308dc78b15ec9b522f7b0f32e7503dea/pkg/protocol/engine/booker/inmemorybooker/booker.go#L150
• We filter blocks in PreSolidFilter and PostSolidFilter based on several rules. However, we don't set blocks as invalid if they are filtered.

Invalid transactions

• we forward blocks with semantically invalid transactions: https://github.com/iotaledger/iota-core/pull/637
• blocks with syntactically invalid transactions are not forwarded as these will not be able to be deserialized correctly as semantic validation happens when we deserialize
• We apply votes of pre-accepted blocks. If anything goes wrong, eg the block votes for two conflicting tx, we don't apply any vote: https://github.com/iotaledger/iota-core/blob/2940df7a25b614f2cde41f609613450da84503c5/pkg/protocol/engine/ledger/ledger/ledger.go#L778

Open questions

• We currently do not mark filtered blocks as invalid. It seems to me that it would make sense to mark such a block as invalid.

[piotrm50]

Blocks are marked as invalid if their transaction is orphaned and the orphaned slot is below the block's slotcommitment slot.

Is that correct though? If transaction is invalid we still book the block carrying that transaction. Why should it be otherwise if the transaction is orphaned when we try to book the block? Shouldn't we treat it the same as an invalid transaction in this case?

A transaction is orphaned when:

• all valid attachments are evicted (then well, the block itself is evicted)
• another spender of a transaction input is committed (the transaction is a losing doublespend).
• one of the inputs is marked as orphaned (which means that the transaction that created it is orphaned as well).

This also seems prone to race conditions, because if one node orphans the transaction, evicts it and then processes a new attachment of that transaction, it would not mark the block as invalid and would successfully attach the transaction to the mempool (as it already forgot about the orphaned version). However if a node evicts slightly later and sees the same block, it would not add a fresh attachments to the mempool (because it still has the orphaned version), and would mark the block (and it's whole future cone) as invalid.

[piotrm50]

Created more specific issues for the above open questions. Closing this one as the core question seems to be clear and not a problem.