walnut-the-cat
commented
2 months ago - Ran a quick experiment to confirm that it reduces witness size significantly: shadow-validate-large-state-values-cache branch
- Discussed the approach with Bowen, decided to consider this along with Shared contract code to avoid duplicate effort later
- Spent the rest of the time trying to come up with a design for MVP, still working on that
Project goal: to ensure that blockchain performance is not bottlenecked around chunk state witness.
Decrease witness size
Decreasing witness size results in faster witness distribution. We already implemented compression as part of the initial release, but more promising ideas described #10780 would potentially result in significantly more size reduction. Dedicated issue: #11911
Optimise distribution
This includes changes on the network side to reduce witness distribution latency. Low level network optimisations such as using UDP instead of TCP requires further investigations and probably significant implementation effort. @saketh-are mentioned that we still have quite some room for improvement with low-level network optimisations while using TCP.
High level optimisations such as changing the way we distribute partial witness can also be considered. In particular we need more robust design around assigning part owners which takes validators stake into account. It was pointed out that this is more of a security/availability concern.
Adjust witness size limits
We might want to do a thorough analysis around witness distribution capabilities and adjust various state witness limits to reflect that more precisely. Currently we have to be very strict with the size limits since chunk resulting in large state witness will block the shard from making further progress. This can changes once we address https://github.com/near/nearcore/issues/11546, so limits can be tuned around average usage instead of extreme edge cases.
Avoid including transactions from both previous and current chunk as part of state witness
Requires further investigations, see #11581.
Exclude contract code from state witness and distribute it separately.
See #11099 for details.