Tests Wishlist for committee based consensus

[GalRogozinski]

Runner

Consensus

• [x] Future decided no instance
• [x] Future decided
• [x] Future Message
• [x] Invalid decided value
• [x] Invalid signature
• [x] Past message
• [x] Post decided
• [x] Post finish
• [x] Valid decided
• [x] Valid decided 7 operators
• [x] Valid decided 10 operators
• [x] Valid message

Duties - New duty

• [x] Consensus not started
• [x] Decided
• [x] Duplicate duty finished
• [x] Duplicate duty not finished
• [x] Finished
• [x] First height
• [x] Not decided
• [x] Post future decided
• [x] Post invalid decided
• [x] Post wrong decided
• [x] Valid

Duties - Attestation Sync Committee

• [x] Starts attestations
• [x] Starts sync committees
• [x] Starts with no duty
• [x] Starts with mixed duties
• [x] Starts maximum possible duties
• [x] Proposal wrong consensus data
• [x] Proposal valid consensus data
• [x] Decided

Post-consensus

• [x] Too many roots
• [x] Too few roots
• [x] Unordered expected roots
• [x] Unknown signer
• [x] Inconsistent beacon signer
• [x] Post finish
• [x] No running duty
• [x] Invalid message signature
• [x] Invalid Beacon signature in quorum
• [x] Duplicate message with different roots
• [x] Duplicate message with different roots then quorum
• [x] Duplicate message
• [x] Invalid expected root
• [x] Pre decided
• [x] Post quorum
• [x] Invalid Message
• [x] Invalid operator signature
• [x] Invalid message slot
• [x] Valid message
• [x] Valid message for 7 operators
• [x] Valid message for 10 operators
• [x] Valid message for 13 operators
• [x] Quorum
• [x] Quorum 7 operators
• [x] Quorum 10 operators
• [x] Quorum 13 operators
• [x] Invalid decided value
• [x] Invalid then quorum
• [x] Invalid quorum then valid quorum
• [x] Inconsistent operator signer
• [x] Invalid decode
• [x] New: wrong validator index

Value check for beacon vote (committee runner's consensus data)

• [x] Valid
• [x] Source higher than target
• [x] Far future target

[GalRogozinski]

For duty tests we skip broadcasted consensus messages: https://github.com/ssvlabs/ssv-spec/blob/548cb4fb149852316f8d46cbf7e62a47a94fafae/ssv/spectest/tests/runner/duties/newduty/test.go#L55-L58

we shouldn't do that... we should test consensus messages as well imo... Maybe it was dont this way to seperate runner from consensus, but we run consensus instance in the test

[GalRogozinski]

Add ValueCheck tests

[GalRogozinski]

We should test the following scenarios:

1. With invalid_quorum_then_valid_quorum we should have byzantine test where some roots have a valid root while others don't. We need to see that a 4th message fixes the invalid ones
2. Test with validators on different committees. We need to make sure the committee indexes are not getting mixed up after post consensus stage.
3. We need to make sure we use proper eth domains.

Currently scenario 3 is implicitly tested because the info is in the broadcasted root (so it is good enough). For 2 we are missing the input.

[GalRogozinski]

Done