Golden tests for persisted StateEvent values

ch1bo commented 2 months ago

Why

We do not want to accidentally break the interface of persisted data and inform users up-front when there is a BREAKING change in the changelog.

What

Adding golden tests to test for potential regressions in the serialization of StateEvent and its constituents. This, plus being vigilant on changes to golden files, allows us to communicate or avoid any breaking changes.

How

By using the ADT variants of hspec-golden-aeson we can ensure all data constructors are covered. See also https://hackage.haskell.org/package/hspec-golden-aeson-0.9.0.0/docs/Test-Aeson-GenericSpecs.html#v:roundtripAndGoldenADTSpecs

This comes at the cost of more golden files, but decreasing sample size to 1 should make this somewhat bearable.

Bonus: this improves and tests some of our generators in hydra-cardano-api

[x] CHANGELOG update not needed
[x] Documentation updated not needed
[x] Haddocks updated
[x] No new TODOs introduced

github-actions[bot] commented 2 months ago

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-09-02 14:04:37.20987018 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff	3799
νCommit	2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d	1743
νHead	bd9fad235c871fb7f837c767593018a84be3083ff80f9dab5f1c55f9	10194
μHead	c8038945816586c4d38926ee63bba67821eb863794220ebbd0bf79ee*	4607

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.
Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5189	5.81	2.30	0.44
2	5389	7.03	2.77	0.46
3	5590	8.56	3.39	0.49
5	5998	11.22	4.43	0.54
10	6999	18.47	7.31	0.66
56	16247	81.43	32.21	1.76

`Commit` transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	559	10.52	4.15	0.29
2	748	13.86	5.65	0.34
3	937	17.33	7.20	0.38
5	1309	24.65	10.44	0.48
10	2238	45.22	19.36	0.75
20	4116	95.99	40.76	1.40

`CollectCom` transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	549	22.14	8.66	0.42
2	114	663	32.96	13.05	0.54
3	170	769	46.33	18.52	0.69
4	226	879	61.38	24.72	0.86
5	283	989	72.63	29.64	0.99
6	340	1100	98.01	39.89	1.28

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	618	17.93	7.87	0.38
2	753	19.11	9.07	0.40
3	916	20.43	10.32	0.43
5	1265	24.36	13.21	0.50
10	2016	31.93	19.81	0.64
50	7661	93.01	72.82	1.76

`Close` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	671	20.98	9.42	0.42
2	774	22.41	10.78	0.44
3	970	24.26	12.52	0.48
5	1262	27.08	15.30	0.53
10	1947	34.85	22.64	0.68
49	8019	98.72	83.15	1.90

`Contest` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	684	27.14	11.67	0.48
2	847	29.24	13.47	0.52
3	1004	31.05	15.02	0.55
5	1315	35.02	18.37	0.62
10	2246	45.61	27.38	0.81
38	6026	95.73	70.50	1.70

`Abort` transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5059	17.43	7.58	0.57
2	5132	27.58	12.01	0.69
3	5363	42.82	18.95	0.87
4	5492	58.44	25.91	1.05
5	5639	77.42	34.46	1.27
6	5812	94.44	42.02	1.47

`FanOut` transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	5022	7.36	3.11	0.45
5	1	57	5057	8.88	3.99	0.47
5	5	285	5192	13.60	6.92	0.54
5	10	569	5362	19.06	10.39	0.62
5	20	1139	5702	30.58	17.60	0.78
5	30	1706	6040	41.70	24.63	0.93
5	40	2275	6379	53.03	31.76	1.09
5	50	2848	6722	64.37	38.89	1.25
5	81	4611	7773	99.53	61.01	1.74

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-09-02 14:06:32.358552265 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	4.640682190
P99	9.154680719999982ms
P95	6.580600049999997ms
P50	4.221868000000001ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	24.000880903
P99	46.91106044000001ms
P95	32.40865785ms
P50	21.946295499999998ms
Number of Invalid txs	0

github-actions[bot] commented 2 months ago

Test Results

489 tests +20 482 :white_check_mark: +20 16m 50s :stopwatch: - 1m 32s 152 suites + 2 7 :zzz: ± 0 5 files ± 0 0 :x: ± 0

Results for commit d1ab629e. ± Comparison against base commit b03fa32f.

This pull request removes 5 and adds 25 tests. Note that renamed tests count towards both.

``` Hydra.Chain.Direct.State/ChainState/JSON encoding of ChainState ‑ allows to encode values with aeson and read them back Hydra.Chain.Direct.State/ChainState/JSON encoding of ChainState ‑ produces the same JSON as is found in golden/ChainState.json Hydra.Chain.Direct.State/Plutus.PubKeyHash/JSON encoding of PubKeyHash ‑ allows to encode values with aeson and read them back Hydra.Chain.Direct.State/Plutus.PubKeyHash/JSON encoding of PubKeyHash ‑ produces the same JSON as is found in golden/PubKeyHash.json Hydra.Ledger.Cardano/PParamss ‑ Roundtrip JSON encoding ``` ``` Hydra.Chain.Direct.State/JSON encoding of PubKeyHash ‑ allows to encode values with aeson and read them back Hydra.Chain.Direct.State/JSON encoding of PubKeyHash ‑ produces the same JSON as is found in golden/PubKeyHash.json Hydra.Events.FileBased/persisted event format/JSON encoding of (StateEvent (Tx BabbageEra)) ‑ allows to encode values with aeson and read them back Hydra.Events.FileBased/persisted event format/JSON encoding of (StateEvent (Tx BabbageEra)) ‑ produces the same JSON as is found in golden/StateEvent (Tx BabbageEra).json Hydra.Events.FileBased/persisted event format/JSON encoding of StateChanged ‑ allows to encode values with aeson and read them back Hydra.Events.FileBased/persisted event format/JSON encoding of StateChanged ‑ produces the same JSON as is found in golden/StateChanged/ChainRolledBack.json Hydra.Events.FileBased/persisted event format/JSON encoding of StateChanged ‑ produces the same JSON as is found in golden/StateChanged/CommittedUTxO.json Hydra.Events.FileBased/persisted event format/JSON encoding of StateChanged ‑ produces the same JSON as is found in golden/StateChanged/DecommitFinalized.json Hydra.Events.FileBased/persisted event format/JSON encoding of StateChanged ‑ produces the same JSON as is found in golden/StateChanged/DecommitRecorded.json Hydra.Events.FileBased/persisted event format/JSON encoding of StateChanged ‑ produces the same JSON as is found in golden/StateChanged/HeadAborted.json … ```

:recycle: This comment has been updated with latest results.

ch1bo commented 2 months ago

@locallycompact You might be delighted to see more genXXX top-level generators :)

cardano-scaling / hydra