Network scalability: 500 parachain validators and 100 cores (async backing enabled)

sandreim commented 1 year ago

Parachain consensus is restricted to a subset of the relay chain validators via themaxValidators configuration parameter. The plan is to remove the parameter, but until then we will gradually increase it's value as we improve the protocol and performance characteristics of the implementation. maxValidators is bumped to higher values only if key performance metrics, specifically parachain block times and finality lag do not degrade when more validators participate in parachain consensus.

The scalability project board is available here: https://github.com/orgs/paritytech/projects/63

After improving our tooling for profiling, gathering metrics and debugging we identified bottlenecks which we are currently addressing.

Experiments and potential improvements are tested on Versi before enabling them on value bearing chains like Kusama and Polkadot. After a battery of tests successfully ran on Versi we have already bumped the number of paravalidators from 200 to 250 on Kusama and we are currently monitoring the key metrics.

Q2 Plan

Q2 milestones:

[x] Increase the number of parachain validators from 200 to 250 (supporting up to 70 parachains) on Kusama
[x] Increase the number of parachain validators from 250 to 300 (supporting up to 70 parachains) on Kusama
[x] Increase the number of parachain validators from 200 to 300 (supporting up 70 parachains) on Polkadot

We've learned from previous tests, that it is already possible for Versi to operate within parameters with 500 paravalidators and 70 parachains, but we've also identified that availability recovery is getting slower as the load on the network I/O increase. Supporting 100 parachains would create even more network load so this needs to be addressed for this milestone.

Q4 Plan

[x] https://github.com/paritytech/polkadot-sdk/issues/620
[x] Investigate any potential availability bottlenecks during load tests and identify potential mitigation using fastpaths
[x] https://github.com/paritytech/polkadot-sdk/issues/702
[x] https://github.com/paritytech/polkadot-sdk/pull/1178
[x] https://github.com/paritytech/polkadot/pull/7214
[x] https://github.com/paritytech/polkadot-sdk/issues/628
[x] https://github.com/paritytech/polkadot-sdk/issues/701
[ ] https://github.com/paritytech/polkadot-sdk/issues/640
[x] https://github.com/paritytech/polkadot/issues/7411
[x] https://github.com/paritytech/polkadot-sdk/issues/577
[ ] https://github.com/paritytech/polkadot-sdk/issues/575
[x] https://github.com/paritytech/polkadot-sdk/pull/1644

Q1 2024 milestone:

[ ] Increase the number of parachain validators from 300 to 500 (supporting up to 100 cores) on Kusama
[ ] Maybe start increase of the active set size on Polkadot.

2024

Polkadot-Forum commented 1 year ago

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-gradual-increase-of-active-validator-set-to-372-validators/3271/1

Polkadot-Forum commented 10 months ago

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/raising-awareness-new-network-validation-protocol-version-v3-coming/5639/1

Polkadot-Forum commented 6 months ago

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/update-validator-set-size-increase-on-kusama/8218/1

burdges commented 6 months ago

At the same time, we might observe a higher approval voting base lag, of up to 1.5 relay chain blocks

We discussed increasing the tranche size or no-show timeout, right?

paritytech / roadmap