feat: generate large state witness

This PR introduces a way to generate arbitrary large chunk state witness as a result of processing a single receipt. This is needed to test what happens with the blockchain if we missed something in the witness size checks. Part of #11656.

Large witness generation is triggered by a function call action with a method name internal_record_storage_garbage_<GARBAGE_SIZE> with <GARBAGE_SIZE> corresponding to a number of megabytes of random data to be added to the storage proof. For example internal_record_storage_garbage_20 would result in 20MB of garbage data in the witness. Having size as part of the method name (and not as an argument) makes it easier to send a transaction using near cli (args needs to be base64 encoded). Note that we don't need to deploy any contracts, failed receipt will still result in garbage added to the witness. This makes it very easy to use, just sending a transaction with a single function call action is enough. The functionality is enabled only when neard is complied with test_features enabled.

Alternative approaches considered:

Adding a new feature that disables witness size checks. Then we can deploy a contract that read/writes a lot of storage data to make the storage proof large. We decided not to proceed with this approach since this is considerably harder to use and also at some point we will still hit compute/gas limits. Another benefit of the chosen approach is that it doesn't increase apply chunk latency, so it allows us to test large witness in isolation from slow chunk application.
11687. The resulting behaviour diverges significantly from what we would expect to happen in the real world when large witness is a result of applying some chunk. For example when testing doomsday scenario we want the next chunk producer to try distributing the same witness after the chunk is missed. This PR also covers the underlying use case #11184.

Large part of this PR is refactoring around runtime tests in chain to make it easier to reuse the existing test setup code.

Codecov Report

Attention: Patch coverage is 97.38220% with 5 lines in your changes missing coverage. Please review.

Project coverage is 71.78%. Comparing base (ced534f) to head (39cd8dc). Report is 15 commits behind head on master.

Files	Patch %	Lines
chain/chain/src/runtime/tests.rs	98.73%	2 Missing :warning:
runtime/runtime/src/actions.rs	86.66%	1 Missing and 1 partial :warning:
core/store/src/trie/mod.rs	93.33%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #11703 +/- ## ========================================== + Coverage 70.19% 71.78% +1.58% ========================================== Files 791 792 +1 Lines 161937 162513 +576 Branches 161937 162513 +576 ========================================== + Hits 113671 116659 +2988 + Misses 43298 40814 -2484 - Partials 4968 5040 +72 ``` | [Flag](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | Coverage Δ | | |---|---|---| | [backward-compatibility](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.23% <0.00%> (-0.01%)` | :arrow_down: | | [db-migration](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.23% <0.00%> (-0.01%)` | :arrow_down: | | [genesis-check](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.35% <0.00%> (-0.01%)` | :arrow_down: | | [integration-tests](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `37.81% <5.23%> (-0.17%)` | :arrow_down: | | [linux](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `71.28% <69.10%> (+1.31%)` | :arrow_up: | | [linux-nightly](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `71.39% <97.38%> (+1.56%)` | :arrow_up: | | [macos](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `54.51% <75.62%> (+1.58%)` | :arrow_up: | | [pytests](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.58% <0.00%> (-0.01%)` | :arrow_down: | | [sanity-checks](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `1.38% <0.00%> (-0.01%)` | :arrow_down: | | [unittests](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `66.20% <97.38%> (+1.74%)` | :arrow_up: | | [upgradability](https://app.codecov.io/gh/near/nearcore/pull/11703/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near) | `0.28% <0.00%> (-0.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=near#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

near / nearcore

feat: generate large state witness #11703

Codecov Report