differentiate seal-validation errors in the protocol state

[AlexHentschel]

Current

• The MutableState interprets any error during seal validation as a byzantine block, even if it is an unexpected exception caused by an internal bug or uncovered edge case
• as this uncovered edge case shows

  {"level":"warn","node_role":"consensus","node_id":"d1cc27f9a94b1b9483759068d55a7229743468f094c03f3ac221b6cad9c114eb","compliance":"core","error":"invalid extension of protocol state (block: a711254d5a5d861d8980451bfdf01af95bca2f0d5b28086f94d8f699ecf1756e, height: 12813441): seal in parent block does not compliance the chain state: seal validation error: unexpected internal error while validating seal 47e7fe53ce508a03eda56d096f1a3999f6a94cd51237cecf607581f81ddbb0cb for result 802aca937528738e671faf1b8d5343d18f921b7a12f483a0f8c93f09df22dfa1 for block d9b451a0ae20f43d5a18d4fce8bb0e5a0b28be8e9e877b2d637a835408014db2: failed to retrieve verifier assignment for result 802aca937528738e671faf1b8d5343d18f921b7a12f483a0f8c93f09df22dfa1 incorporated in block 6d2ebf5fa921caa5b9f44e4a566c76a4490d4b9b3c9ce509e6d39f0e12afb8b4: failed to retrieve source of randomness: failed to get valid child of block 6d2ebf5fa921caa5b9f44e4a566c76a4490d4b9b3c9ce509e6d39f0e12afb8b4: block has no valid children (total children: 1)","suspicious":true,"time":"2022-11-14T23:07:00.356961956Z","message":"received invalid block from other node (potential slashing evidence?)"}

  received invalid block from other node (potential slashing evidence?)

In the mature system, a node affected by this edge case would raise a false slashing challenge, resulting in the node itself being slashed. This demonstrates the importance of carefully differentiating errors types and avoiding blanked associations of all errors with a single cause.

Suggestions

• move the error documentation to the interface's Validate method and also include it in Validate implementation
• refactor MutableState to implement proper error handling according to our Coding Style Guidelines
• re-add suspicious log flag to InvalidInput error handling (see discussion https://github.com/onflow/flow-go/pull/3589#discussion_r1026565769)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.