Add workflow to bump hydra spec flake input

ch1bo commented 3 weeks ago

This will create a PR whenever the default branch of https://github.com/cardano-scaling/hydra-formal-specification was updated.

[x] CHANGELOG update not needed
[x] Documentation update not needed
[x] Haddocks update not needed
[x] No new TODOs introduced

github-actions[bot] commented 3 weeks ago

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-08-22 16:08:34.549152488 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff	3799
νCommit	2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d	1743
νHead	bd9fad235c871fb7f837c767593018a84be3083ff80f9dab5f1c55f9	10194
μHead	c8038945816586c4d38926ee63bba67821eb863794220ebbd0bf79ee*	4607

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.
Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5190	5.88	2.33	0.44
2	5387	6.99	2.76	0.46
3	5595	8.56	3.39	0.49
5	5994	11.22	4.43	0.54
10	6999	18.43	7.30	0.66
56	16246	81.53	32.25	1.76

`Commit` transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	559	10.52	4.15	0.29
2	746	13.86	5.65	0.34
3	931	17.33	7.20	0.38
5	1307	24.65	10.44	0.48
10	2243	45.22	19.36	0.75
20	4114	95.99	40.76	1.40

`CollectCom` transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	553	21.49	8.43	0.41
2	114	659	32.96	13.05	0.54
3	171	769	44.12	17.70	0.67
4	225	879	61.58	24.80	0.87
5	282	989	79.22	32.06	1.06
6	338	1100	96.34	39.27	1.26

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	642	17.71	7.80	0.38
2	823	20.26	9.49	0.42
3	904	20.00	10.08	0.42
5	1274	25.52	13.72	0.51
10	2097	35.21	21.11	0.68
50	7717	94.97	73.53	1.79

`Close` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	596	20.56	9.05	0.41
2	825	22.82	11.13	0.45
3	916	23.88	12.19	0.47
5	1217	27.24	15.32	0.53
10	2016	35.50	23.10	0.69
49	7962	98.23	82.73	1.89

`Contest` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	690	27.18	11.69	0.48
2	811	28.93	13.18	0.51
3	891	30.26	14.34	0.54
5	1268	34.44	17.97	0.61
10	1859	43.10	25.26	0.76
39	6506	99.74	73.87	1.77

`Abort` transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5055	17.36	7.55	0.57
2	5132	27.48	11.97	0.68
3	5317	38.80	17.04	0.82
4	5445	53.83	23.76	1.00
5	5665	77.58	34.53	1.28

`FanOut` transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	5022	7.95	3.36	0.46
5	1	57	5056	8.69	3.91	0.47
5	5	284	5192	13.21	6.75	0.53
5	10	569	5361	19.26	10.48	0.62
5	20	1138	5701	30.38	17.51	0.77
5	30	1706	6040	41.90	24.72	0.93
5	40	2277	6382	52.84	31.67	1.09
5	50	2847	6721	64.37	38.89	1.25
5	81	4609	7770	99.53	61.01	1.74

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-08-22 16:11:06.36997249 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	5.059109541
P99	9.661105769999812ms
P95	6.735598649999998ms
P50	4.472921ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	23.725055880
P99	93.16925251000188ms
P95	31.716170749999996ms
P50	21.415048ms
Number of Invalid txs	0

github-actions[bot] commented 3 weeks ago

Test Results

469 tests ±0 462 :white_check_mark: ±0 18m 20s :stopwatch: + 1m 21s 150 suites ±0 7 :zzz: ±0 5 files ±0 0 :x: ±0

Results for commit 00862617. ± Comparison against base commit 16c87520.

locallycompact commented 3 weeks ago

This is cool. I would still prefer to have it be of the fashion that we do not import spec changes that the code does not reflect, but for most updates to the spec like fixes or refactors that don't change the functionality then this is cool.

If we could reach a point where the spec is generating some kind of test for us downstream, then the above property could be enforced by the CI too.

ch1bo commented 3 weeks ago

I would still prefer to have it be of the fashion that we do not import spec changes that the code does not reflect, but for most updates to the spec like fixes or refactors that don't change the functionality then this is cool.

@locallycompact I thought we could do this for a sanity check at first. So we don't miss out on any spec changes accidentally. Maybe it's annoying and we remove this workflow again if we have spec changes that are not implemented for long.

cardano-scaling / hydra