paritytech / polkadot-sdk

The Parity Polkadot Blockchain SDK
https://polkadot.network/
1.88k stars 685 forks source link

Zombienet testing for validator disabling #1590

Open tdimitrov opened 1 year ago

tdimitrov commented 1 year ago

The tests should validate that validator disabling and the disabling strategy behaves correctly.

Base test cases to cover:

Note: THRESHOLD is an implementation parameter representing max number of validators we can have disabled at each point in time.

Part of https://github.com/paritytech/polkadot-sdk/issues/784

ordian commented 1 year ago

1841 adds a basic zombienet test which only checks for the log line about disabled state. This is not exactly what we want, which is to test the effects of disabling:

this is a bit tricky to test with the current zombienet v1 as the .zndsl framework is quite limited and not a (turing-complete) language. We will need an ability to assert that some metrics hasn't increased after some (unknown at compile time) point. I believe this will be addressed with zombienet v2 (cc @pepoviola)

pepoviola commented 11 months ago

1841 adds a basic zombienet test which only checks for the log line about disabled state. This is not exactly what we want, which is to test the effects of disabling:

  • once a (malus) validator initiated a dispute that concluded against them, it should no longer be able to initiate disputes and their backing statements should be ignored

this is a bit tricky to test with the current zombienet v1 as the .zndsl framework is quite limited and not a (turing-complete) language. We will need an ability to assert that some metrics hasn't increased after some (unknown at compile time) point. I believe this will be addressed with zombienet v2 (cc @pepoviola)

Hey @ordian, sorry about the delay. We expect to have the new version ready for start moving some workload by the end of next week. We can start exploring this case with the native provider now and move to k8s in the near future. Thx!