[Testing] Remove non-determinism from E2E Consensus Tests - Githubissues

pokt-network / pocket

Official implementation of the Pocket Network Protocol v1

https://pokt.network

MIT License

61 stars 33 forks source link

[Testing] Remove non-determinism from E2E Consensus Tests #959

Open Olshansk opened 11 months ago

Olshansk commented 11 months ago

Objective

Avoid using timers or other non-deterministic approaches in E2E tests.

Origin Document

The following comment in #874 and as it relates to https://github.com/pokt-network/pocket/pull/948/files#diff-01dec4121ae8acb7a1f4bb72a6c2104827d2c2d2197eb5f45fa5c032ffba32cdR93:

Screenshot 2023-08-01 at 4 29 47 PM

Goals

Prevent non-determinism and flakiness from E2E tests
Create robust tooling to help iterate on E2E tests

Deliverable

[ ] Remove all the developer waits for ... milliseconds in all .feature files
[ ] Introduce alternate mechanisms mechanism to that waiting for a node to reach a certain height include and not limited to:
- [ ] Polling for a specific height (with a max timeout)
- [ ] Polling a health check endpoint (with a max timeout)

Non-goals / Non-deliverables

Introducing new tests
Introducing new functionality

General issue deliverables

[ ] Update any relevant local/global README(s)
[ ] Add or update any relevant or supporting mermaid diagrams

Testing Methodology

[ ] All tests: make test_all
[ ] E2E Tests: make test_e2e
[ ] LocalNet: verify a LocalNet is still functioning correctly by following the instructions at docs/development/README.md
[ ] k8s LocalNet: verify a k8s LocalNet is still functioning correctly by following the instructions here

Creator: @Olshansk Co-Owners: @0xBigBoss @dylanlott

Olshansk commented 11 months ago

Copying kubectl wait --for=condition=ready pod/full-node-001-pocket-0 here so the idea is not based on this comment:

Screenshot 2023-08-03 at 11 56 01 AM