search

cardano-scaling / hydra

Implementation of the Hydra Head protocol
https://hydra.family/head-protocol/
Apache License 2.0
284 stars 86 forks source link

Display difference in tx-cost outputs as a comment on PRs #1703

Closed noonio closed 1 month ago

noonio commented 1 month ago

This adds a simple diff calculation to the PR comment, just like the present "Transaction costs" comment we get, but instead a diff between the PR and the master branch.

Positive numbers are bad - it means the cost/size/etc got larger with the introduction of the PR. These are coloured black.

Negative numbers are good - it means a reduction in size/cost/etc, so these are coloured green.

Values that are the same between PR and branch are simply listed as -; i.e. unchanged.

[!note] It seems that some differences are expected even when nothing relevant has changed, as in this PR itself. So some care will need to be taken in learning how to interpret these numbers, as we see them develop over furture PRs.

github-actions[bot] commented 1 month ago

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2024-10-14 06:11:14.14569368 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial b512161ccb0652d7e9a0b540e4a3c808f73d6558a4bcabf374d85880 3969
νCommit ea444d37d226e71eef73ac78d149750da977feb588900135bf9e8221 692
νHead 2253ddd95837c7aacc8635a971caaea743434152dd8dd2849bdf4162 10797
μHead 4d648ca239040b0e87901835aa11423e7aa3bd947ce6befe7db1bae8* 4508
νDeposit 1a011f23b139a6426767026bde10319546485d553219a5848cdac4e5 2993
Parties Tx size % max Mem % max CPU Min fee ₳
1 5094 5.61 2.21 0.44
2 5295 7.13 2.82 0.46
3 5500 8.55 3.38 0.49
5 5902 11.32 4.48 0.53
10 6908 18.10 7.16 0.65
57 16353 83.00 32.84 1.78

Commit transaction costs

This uses ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 569 10.84 4.26 0.29
2 758 14.31 5.80 0.34
3 947 17.92 7.39 0.39
5 1322 25.56 10.73 0.49
10 2257 47.11 19.97 0.77
19 3938 94.71 39.81 1.38

CollectCom transaction costs

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 57 560 19.87 7.59 0.39
2 113 671 28.85 10.97 0.49
3 170 782 36.31 13.81 0.58
4 226 893 47.09 17.87 0.70
5 282 1004 53.50 20.33 0.77
6 338 1116 61.35 23.32 0.86
7 393 1227 77.06 29.18 1.04
8 452 1342 72.99 27.80 1.00
9 506 1449 78.52 29.96 1.06
10 560 1564 86.68 33.07 1.16

Cost of Decrement Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 657 18.68 8.18 0.39
2 757 19.35 9.15 0.41
3 968 21.52 10.75 0.44
5 1240 24.37 13.42 0.50
10 2010 33.33 20.76 0.66
48 7742 97.22 74.94 1.82

Close transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 644 20.83 9.31 0.41
2 809 22.63 11.06 0.45
3 975 24.11 12.53 0.48
5 1320 27.88 16.09 0.55
10 1977 35.12 23.28 0.69
50 8133 99.89 86.55 1.94

Contest transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 679 26.81 11.48 0.48
2 842 28.53 13.04 0.51
3 943 30.35 14.59 0.54
5 1356 34.85 18.54 0.63
10 2012 43.17 26.03 0.78
39 6499 99.84 75.77 1.79

Abort transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 4985 15.43 6.61 0.54
2 5111 21.73 9.35 0.62
3 5207 28.53 12.30 0.70
4 5455 36.29 15.85 0.80
5 5463 39.85 17.18 0.84
6 5574 45.78 19.78 0.91
7 5681 55.42 24.00 1.02
8 6112 65.81 28.92 1.16
9 6178 71.89 31.54 1.23
10 6178 74.93 32.68 1.26
11 6438 86.07 37.71 1.40
12 6474 89.13 38.84 1.44
13 6625 97.52 42.53 1.54
15 6498 99.71 42.70 1.55

FanOut transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
10 0 0 5089 10.19 4.26 0.49
10 1 57 5123 11.16 4.90 0.50
10 5 284 5259 15.83 7.79 0.57
10 10 568 5428 21.84 11.48 0.65
10 20 1139 5768 33.16 18.55 0.81
10 30 1710 6111 45.21 25.95 0.97
10 40 2277 6448 57.02 33.24 1.13
10 50 2848 6789 68.63 40.45 1.30
10 76 4324 7667 98.68 59.14 1.71

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-10-14 06:12:54.34733395 UTC

Baseline Scenario

Number of nodes 1
Number of txs 300
Avg. Confirmation Time (ms) 5.222795416
P99 10.082973439999995ms
P95 6.651513650000001ms
P50 4.9907825ms
Number of Invalid txs 0

Three local nodes

Number of nodes 3
Number of txs 900
Avg. Confirmation Time (ms) 22.766152485
P99 116.95122445999998ms
P95 30.358121399999998ms
P50 20.231026ms
Number of Invalid txs 0
github-actions[bot] commented 1 month ago

Transaction cost differences

Script summary

Name Size (Bytes)
νInitial -
νCommit -
νHead -
μHead -
νDeposit -

Init transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 +7 $${\color{green}-0.06}$$ $${\color{green}-0.02}$$ -
2 $${\color{green}-1.00}$$ $${\color{green}-0.03}$$ $${\color{green}-0.01}$$ -
3 +7 $${\color{green}-0.09}$$ $${\color{green}-0.04}$$ -
5 $${\color{green}-1.00}$$ +0.19 +0.09 -
10 +3 - - -
57 - $${\color{green}-0.10}$$ $${\color{green}-0.04}$$ -

Commit transaction costs

UTxO Tx size % max Mem % max CPU Min fee ₳
1 - - - -
2 - - - -
3 +3 - - -
5 +2 - - -
10 +7 - - -
19 - - - -

CollectCom transaction costs

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 - $${\color{green}-4.00}$$ - - -
2 $${\color{green}-1.00}$$ - $${\color{green}-0.91}$$ $${\color{green}-0.33}$$ $${\color{green}-0.01}$$
3 - - +3.94 +1.46 +0.04
4 - - +5.45 +2.0 +0.06
5 +1.0 - $${\color{green}-1.89}$$ $${\color{green}-0.71}$$ $${\color{green}-0.02}$$
6 +1.0 - +8.97 +3.29 +0.09
7 +3.0 - $${\color{green}-10.58}$$ $${\color{green}-3.92}$$ $${\color{green}-0.12}$$
8 $${\color{green}-2.00}$$ - +8.11 +2.94 +0.09
9 +1.0 - +9.52 +3.58 +0.1

Cost of Decrement Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 $${\color{green}-19.00}$$ +0.28 +0.09 -
2 +71 +1.09 +0.41 +0.02
3 $${\color{green}-46.00}$$ $${\color{green}-0.68}$$ $${\color{green}-0.25}$$ $${\color{green}-0.01}$$
5 $${\color{green}-127.00}$$ $${\color{green}-2.48}$$ $${\color{green}-1.00}$$ $${\color{green}-0.04}$$
10 +93 $${\color{green}-0.85}$$ $${\color{green}-0.39}$$ -
50 $${\color{green}-171.00}$$ $${\color{green}-2.92}$$ $${\color{green}-1.11}$$ $${\color{green}-0.04}$$

Close transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 - $${\color{green}-0.04}$$ $${\color{green}-0.02}$$ -
2 +46 +0.37 +0.32 +0.01
3 $${\color{green}-4.00}$$ - - -
5 $${\color{green}-74.00}$$ $${\color{green}-0.41}$$ $${\color{green}-0.37}$$ -
10 $${\color{green}-44.00}$$ +0.15 +0.02 -
50 +274 +1.56 +1.68 +0.04

Contest transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 +29.0 +0.05 +0.04 -
2 $${\color{green}-45.00}$$ $${\color{green}-0.45}$$ $${\color{green}-0.35}$$ $${\color{green}-0.01}$$
3 $${\color{green}-90.00}$$ $${\color{green}-0.32}$$ $${\color{green}-0.34}$$ $${\color{green}-0.01}$$
5 $${\color{green}-133.00}$$ $${\color{green}-1.11}$$ $${\color{green}-0.97}$$ $${\color{green}-0.03}$$
10 +333.0 +1.75 +1.67 +0.05

Abort transaction costs

Parties Tx size % max Mem % max CPU Min fee ₳
1 +81 +2.86 +1.47 +0.04
2 +41 +2.62 +1.26 +0.03
3 $${\color{green}-22.00}$$ $${\color{green}-1.85}$$ $${\color{green}-0.87}$$ $${\color{green}-0.02}$$
4 $${\color{green}-132.00}$$ $${\color{green}-4.31}$$ $${\color{green}-2.13}$$ $${\color{green}-0.06}$$
5 $${\color{green}-125.00}$$ $${\color{green}-6.16}$$ $${\color{green}-3.01}$$ $${\color{green}-0.08}$$
6 +209 +5.06 +2.56 +0.07
7 +108 +6.48 +3.13 +0.08
8 $${\color{green}-191.00}$$ $${\color{green}-0.93}$$ $${\color{green}-0.62}$$ $${\color{green}-0.02}$$
9 +82 +0.54 +0.39 +0.01
10 $${\color{green}-201.00}$$ $${\color{green}-8.18}$$ $${\color{green}-3.93}$$ $${\color{green}-0.10}$$
11 +230 $${\color{green}-0.95}$$ $${\color{green}-0.16}$$ -
12 $${\color{green}-26.00}$$ +4.63 +1.9 +0.05

FanOut transaction costs

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
10 - - - $${\color{green}-0.59}$$ $${\color{green}-0.25}$$ $${\color{green}-0.01}$$
10 - - $${\color{green}-1.00}$$ +0.39 +0.17 -
10 - - - +1.08 +0.46 +0.01
10 - $${\color{green}-2.00}$$ $${\color{green}-2.00}$$ +0.2 +0.08 -
10 - - - +0.67 +0.29 -
10 - +5 +5 +0.2 +0.09 -
10 - $${\color{green}-3.00}$$ $${\color{green}-3.00}$$ +0.2 +0.09 +0.01
10 - +3 +3 +0.08 +0.04 -
10 +1 +55 +32 +0.49 +0.43 +0.01
github-actions[bot] commented 1 month ago

Test Results

544 tests  ±0   538 :white_check_mark: ±0   26m 30s :stopwatch: +45s 162 suites ±0     6 :zzz: ±0    7 files   ±0     0 :x: ±0 

Results for commit 90f61d42. ± Comparison against base commit d8c0a498.

:recycle: This comment has been updated with latest results.