In order to increase robustness of the network, we should make sure it can withstand long finality stalls without issues like running out of memory. The slowdown of block production is definitely already a very good measure in place to limit growth, nevertheless it is worth checking, with certain contrived test loads we certainly were able to increase memory usage significantly.
As a first check, we could modify the source to just not finalize, then keep a test network running with this setup for a while and monitor memory usage. Ideally we have some metrics to check memory usage on individual subsystems, so we can immediately narrow down issues.
A variant of above check would be to have a moderate number of disputes happening as well, otherwise we would hardly notice any problems in dispute subsystems.
Due to the block production slowdown this should hardly become an issue in practice, hence I would consider this ticket low priority - just something to make Polkadot as rock solid as possible, once we ironed out other more important stuff. That said, having the above as an automated Zombienet test would make a lot of sense, so we become aware of regressions immediately.
eskimor
commented
2 years ago
In order to increase robustness of the network, we should make sure it can withstand long finality stalls without issues like running out of memory. The slowdown of block production is definitely already a very good measure in place to limit growth, nevertheless it is worth checking, with certain contrived test loads we certainly were able to increase memory usage significantly.
As a first check, we could modify the source to just not finalize, then keep a test network running with this setup for a while and monitor memory usage. Ideally we have some metrics to check memory usage on individual subsystems, so we can immediately narrow down issues.
A variant of above check would be to have a moderate number of disputes happening as well, otherwise we would hardly notice any problems in dispute subsystems.
Due to the block production slowdown this should hardly become an issue in practice, hence I would consider this ticket low priority - just something to make Polkadot as rock solid as possible, once we ironed out other more important stuff. That said, having the above as an automated Zombienet test would make a lot of sense, so we become aware of regressions immediately.