Automated Replay Testing (Chicken Dance) - MVP - Githubissues

eosnetworkfoundation / product

A workspace for product management primarily for planning purposes

2 stars 3 forks source link

Automated Replay Testing (Chicken Dance) - MVP #91

Open stephenpdeos opened 1 year ago

stephenpdeos commented 1 year ago

Short Description

As an Antelope Core developers, I want to be able to run an automated replay test from genesis within 1 day, to confirm that consensus is maintained and uptime reliability continues.

What

To verify that the consensus mechanism is intact, we can run a full replay from genesis and confirm that every block can be applied without error, and that we achieve the expected state after replay. It may be valuable to run multiple replay tests with different options (for example: supported CPU features, using various WASM runtimes). This would allow us to gain confidence that consensus is robust under many different environmental situations.

Why

Running a full replay from genesis manually would be a months-long endeavor. We need to be able to run a replay test within 1 day in an automated and repeatable fashion.

Considerations

These tests may be expensive to run
If developed correctly, it should be trivial to add other Antelope chains

Requirements

A developer can "hit a button" to launch a replay test for a particular commit, and see if completed within a day.
Ability to specify command line options for test (including WASM runtime)

Non-requirements (Potential Future Version of Chicken Dance)

Ability to easily run with history from other Antelope chains (Jungle, WAX, TELOS, etc)
Ability to run on different hardware to simulate a variety of environmental factors (CPU features)
```
### Tasks
```
[ ] https://github.com/AntelopeIO/leap/issues/1353
[ ] https://github.com/AntelopeIO/leap/issues/1419
[ ] https://github.com/AntelopeIO/leap/issues/1420
[ ] https://github.com/AntelopeIO/leap/issues/1421
[ ] https://github.com/AntelopeIO/leap/issues/1441
[ ] https://github.com/AntelopeIO/leap/issues/1422
[ ] https://github.com/AntelopeIO/leap/issues/1423
[ ] https://github.com/AntelopeIO/leap/issues/1427
[ ] https://github.com/AntelopeIO/leap/issues/1429
[ ] https://github.com/AntelopeIO/leap/issues/1430
[ ] https://github.com/AntelopeIO/leap/issues/1431
[ ] https://github.com/AntelopeIO/leap/issues/1426
[ ] https://github.com/AntelopeIO/leap/issues/1424
[ ] https://github.com/eosnetworkfoundation/replay-test/issues/14

bhazzard commented 1 year ago

@BenjaminGormanPMP Next step is task breakdown.

COULD depend on Pinned Builds (but doesn't have to, it would just be ideal).

bhazzard commented 1 year ago

@bhazzard Create issue for Solution Design

bhazzard commented 1 year ago

@bhazzard Create issue for Solution Design

Done: https://github.com/AntelopeIO/leap/issues/1353

bhazzard commented 1 year ago

Relevant past work: https://hackmd.io/DFtefcHjRoGxU03N_86g0g