IntersectMBO / plutus

The Plutus language implementation and tools
Apache License 2.0
1.57k stars 479 forks source link

[Benchmark] Include 'mkTermToEvaluate' in 'benchmark-decode' #6667

Open effectfully opened 1 week ago

effectfully commented 1 week ago

mkTermToEvaluate is a part of the script preparation phase. It looks rather inefficient (in two different ways) and we don't seem to have any dedicated benchmarks for catching that other than validation-full, but that's too insensitive due to it evaluating the scripts. I really don't believe there's a good reason for mkTermToEvaluate to be inefficient, so I think we should include it in the decoding benchmarks.

Let's see how much overhead mkTermToEvaluate contributes.

effectfully commented 1 week ago

/benchmark validation-decode

effectfully commented 1 week ago

/benchmark validation-decode

effectfully commented 1 week ago

/benchmark validation-decode

github-actions[bot] commented 1 week ago

Click here to check the status of your benchmark.

github-actions[bot] commented 1 week ago

Comparing benchmark results of 'validation-decode' on '1d9a758c1f' (base) and 'a803b77a84' (PR)

Results table | Script | 1d9a758 | a803b77 | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 276.4 μs | 336.3 μs | +21.7% | | auction_1-2 | 778.3 μs | 977.7 μs | +25.6% | | auction_1-3 | 778.6 μs | 968.2 μs | +24.4% | | auction_1-4 | 276.6 μs | 335.5 μs | +21.3% | | auction_2-1 | 275.9 μs | 335.5 μs | +21.6% | | auction_2-2 | 761.1 μs | 966.3 μs | +27.0% | | auction_2-3 | 779.0 μs | 952.2 μs | +22.2% | | auction_2-4 | 777.5 μs | 964.1 μs | +24.0% | | auction_2-5 | 276.9 μs | 335.0 μs | +21.0% | | crowdfunding-success-1 | 337.9 μs | 412.2 μs | +22.0% | | crowdfunding-success-2 | 337.8 μs | 412.6 μs | +22.1% | | crowdfunding-success-3 | 337.3 μs | 412.0 μs | +22.1% | | currency-1 | 341.3 μs | 409.5 μs | +20.0% | | escrow-redeem_1-1 | 450.9 μs | 555.0 μs | +23.1% | | escrow-redeem_1-2 | 450.2 μs | 553.8 μs | +23.0% | | escrow-redeem_2-1 | 447.5 μs | 554.2 μs | +23.8% | | escrow-redeem_2-2 | 450.9 μs | 553.5 μs | +22.8% | | escrow-redeem_2-3 | 449.2 μs | 554.9 μs | +23.5% | | escrow-refund-1 | 449.0 μs | 552.6 μs | +23.1% | | future-increase-margin-1 | 341.3 μs | 410.8 μs | +20.4% | | future-increase-margin-2 | 454.6 μs | 559.5 μs | +23.1% | | future-increase-margin-3 | 455.5 μs | 560.6 μs | +23.1% | | future-increase-margin-4 | 979.9 μs | 1.199 ms | +22.4% | | future-increase-margin-5 | 968.6 μs | 1.194 ms | +23.3% | | future-pay-out-1 | 340.2 μs | 408.8 μs | +20.2% | | future-pay-out-2 | 455.8 μs | 560.5 μs | +23.0% | | future-pay-out-3 | 452.8 μs | 558.7 μs | +23.4% | | future-pay-out-4 | 979.0 μs | 1.200 ms | +22.6% | | future-settle-early-1 | 341.0 μs | 408.3 μs | +19.7% | | future-settle-early-2 | 453.2 μs | 559.5 μs | +23.5% | | future-settle-early-3 | 452.6 μs | 560.7 μs | +23.9% | | future-settle-early-4 | 959.8 μs | 1.199 ms | +24.9% | | game-sm-success_1-1 | 754.8 μs | 924.9 μs | +22.5% | | game-sm-success_1-2 | 234.3 μs | 288.3 μs | +23.0% | | game-sm-success_1-3 | 755.3 μs | 923.9 μs | +22.3% | | game-sm-success_1-4 | 233.4 μs | 288.3 μs | +23.5% | | game-sm-success_2-1 | 755.6 μs | 922.7 μs | +22.1% | | game-sm-success_2-2 | 233.7 μs | 287.9 μs | +23.2% | | game-sm-success_2-3 | 737.3 μs | 923.7 μs | +25.3% | | game-sm-success_2-4 | 235.0 μs | 287.9 μs | +22.5% | | game-sm-success_2-5 | 747.0 μs | 923.4 μs | +23.6% | | game-sm-success_2-6 | 234.4 μs | 287.6 μs | +22.7% | | multisig-sm-1 | 823.7 μs | 1.029 ms | +24.9% | | multisig-sm-2 | 839.6 μs | 1.027 ms | +22.3% | | multisig-sm-3 | 840.8 μs | 1.028 ms | +22.3% | | multisig-sm-4 | 839.5 μs | 1.027 ms | +22.3% | | multisig-sm-5 | 838.5 μs | 1.029 ms | +22.7% | | multisig-sm-6 | 839.5 μs | 1.029 ms | +22.6% | | multisig-sm-7 | 828.0 μs | 1.029 ms | +24.3% | | multisig-sm-8 | 839.4 μs | 1.027 ms | +22.3% | | multisig-sm-9 | 840.5 μs | 1.028 ms | +22.3% | | multisig-sm-10 | 841.0 μs | 1.030 ms | +22.5% | | ping-pong-1 | 706.0 μs | 862.5 μs | +22.2% | | ping-pong-2 | 704.0 μs | 861.0 μs | +22.3% | | ping-pong_2-1 | 690.7 μs | 859.2 μs | +24.4% | | prism-1 | 230.1 μs | 279.8 μs | +21.6% | | prism-2 | 750.9 μs | 913.2 μs | +21.6% | | prism-3 | 344.5 μs | 415.6 μs | +20.6% | | pubkey-1 | 236.2 μs | 288.5 μs | +22.1% | | stablecoin_1-1 | 1.258 ms | 1.521 ms | +20.9% | | stablecoin_1-2 | 234.1 μs | 288.8 μs | +23.4% | | stablecoin_1-3 | 1.261 ms | 1.523 ms | +20.8% | | stablecoin_1-4 | 235.0 μs | 286.9 μs | +22.1% | | stablecoin_1-5 | 1.257 ms | 1.525 ms | +21.3% | | stablecoin_1-6 | 233.7 μs | 288.2 μs | +23.3% | | stablecoin_2-1 | 1.257 ms | 1.521 ms | +21.0% | | stablecoin_2-2 | 234.8 μs | 288.6 μs | +22.9% | | stablecoin_2-3 | 1.231 ms | 1.524 ms | +23.8% | | stablecoin_2-4 | 234.4 μs | 287.4 μs | +22.6% | | token-account-1 | 339.8 μs | 412.8 μs | +21.5% | | token-account-2 | 303.4 μs | 372.0 μs | +22.6% | | uniswap-1 | 336.2 μs | 417.8 μs | +24.3% | | uniswap-2 | 331.3 μs | 409.6 μs | +23.6% | | uniswap-3 | 1.031 ms | 1.274 ms | +23.6% | | uniswap-4 | 255.0 μs | 311.0 μs | +22.0% | | uniswap-5 | 1.030 ms | 1.284 ms | +24.7% | | uniswap-6 | 254.9 μs | 309.0 μs | +21.2% | | vesting-1 | 456.9 μs | 566.1 μs | +23.9% |
1d9a758 a803b77 Change
TOTAL 45.39 ms 55.75 ms +22.8%
github-actions[bot] commented 1 week ago

Click here to check the status of your benchmark.

github-actions[bot] commented 1 week ago

Comparing benchmark results of 'validation-decode' on '1d9a758c1f' (base) and 'a803b77a84' (PR)

Results table | Script | 1d9a758 | a803b77 | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 266.6 μs | 328.2 μs | +23.1% | | auction_1-2 | 756.0 μs | 940.1 μs | +24.4% | | auction_1-3 | 758.0 μs | 940.0 μs | +24.0% | | auction_1-4 | 267.5 μs | 328.1 μs | +22.7% | | auction_2-1 | 267.6 μs | 329.1 μs | +23.0% | | auction_2-2 | 737.1 μs | 938.6 μs | +27.3% | | auction_2-3 | 756.8 μs | 938.0 μs | +23.9% | | auction_2-4 | 756.9 μs | 941.3 μs | +24.4% | | auction_2-5 | 267.0 μs | 328.4 μs | +23.0% | | crowdfunding-success-1 | 326.3 μs | 405.1 μs | +24.1% | | crowdfunding-success-2 | 326.0 μs | 405.1 μs | +24.3% | | crowdfunding-success-3 | 325.2 μs | 405.4 μs | +24.7% | | currency-1 | 328.4 μs | 403.0 μs | +22.7% | | escrow-redeem_1-1 | 433.9 μs | 543.8 μs | +25.3% | | escrow-redeem_1-2 | 432.7 μs | 546.2 μs | +26.2% | | escrow-redeem_2-1 | 436.0 μs | 546.1 μs | +25.3% | | escrow-redeem_2-2 | 432.9 μs | 543.3 μs | +25.5% | | escrow-redeem_2-3 | 433.2 μs | 543.7 μs | +25.5% | | escrow-refund-1 | 432.4 μs | 543.9 μs | +25.8% | | future-increase-margin-1 | 329.5 μs | 403.1 μs | +22.3% | | future-increase-margin-2 | 439.3 μs | 549.0 μs | +25.0% | | future-increase-margin-3 | 437.6 μs | 550.7 μs | +25.8% | | future-increase-margin-4 | 940.0 μs | 1.181 ms | +25.6% | | future-increase-margin-5 | 947.6 μs | 1.183 ms | +24.8% | | future-pay-out-1 | 328.6 μs | 401.6 μs | +22.2% | | future-pay-out-2 | 437.5 μs | 549.3 μs | +25.6% | | future-pay-out-3 | 439.6 μs | 549.6 μs | +25.0% | | future-pay-out-4 | 950.4 μs | 1.182 ms | +24.4% | | future-settle-early-1 | 328.8 μs | 400.6 μs | +21.8% | | future-settle-early-2 | 439.5 μs | 552.3 μs | +25.7% | | future-settle-early-3 | 437.1 μs | 549.6 μs | +25.7% | | future-settle-early-4 | 929.2 μs | 1.182 ms | +27.2% | | game-sm-success_1-1 | 725.6 μs | 911.5 μs | +25.6% | | game-sm-success_1-2 | 227.2 μs | 280.8 μs | +23.6% | | game-sm-success_1-3 | 733.0 μs | 911.3 μs | +24.3% | | game-sm-success_1-4 | 231.7 μs | 281.6 μs | +21.5% | | game-sm-success_2-1 | 734.2 μs | 909.3 μs | +23.8% | | game-sm-success_2-2 | 227.1 μs | 281.6 μs | +24.0% | | game-sm-success_2-3 | 732.9 μs | 910.9 μs | +24.3% | | game-sm-success_2-4 | 226.8 μs | 283.4 μs | +25.0% | | game-sm-success_2-5 | 731.9 μs | 913.8 μs | +24.9% | | game-sm-success_2-6 | 226.5 μs | 283.6 μs | +25.2% | | multisig-sm-1 | 816.8 μs | 1.018 ms | +24.6% | | multisig-sm-2 | 816.2 μs | 1.018 ms | +24.7% | | multisig-sm-3 | 815.9 μs | 1.015 ms | +24.4% | | multisig-sm-4 | 817.2 μs | 1.017 ms | +24.4% | | multisig-sm-5 | 816.1 μs | 1.020 ms | +25.0% | | multisig-sm-6 | 813.9 μs | 1.019 ms | +25.2% | | multisig-sm-7 | 817.0 μs | 1.015 ms | +24.2% | | multisig-sm-8 | 815.1 μs | 1.016 ms | +24.6% | | multisig-sm-9 | 814.4 μs | 1.018 ms | +25.0% | | multisig-sm-10 | 813.9 μs | 1.016 ms | +24.8% | | ping-pong-1 | 683.3 μs | 850.7 μs | +24.5% | | ping-pong-2 | 683.3 μs | 847.0 μs | +24.0% | | ping-pong_2-1 | 683.7 μs | 850.1 μs | +24.3% | | prism-1 | 222.8 μs | 275.2 μs | +23.5% | | prism-2 | 726.7 μs | 900.2 μs | +23.9% | | prism-3 | 333.8 μs | 410.0 μs | +22.8% | | pubkey-1 | 227.4 μs | 282.4 μs | +24.2% | | stablecoin_1-1 | 1.221 ms | 1.504 ms | +23.2% | | stablecoin_1-2 | 225.5 μs | 281.7 μs | +24.9% | | stablecoin_1-3 | 1.221 ms | 1.508 ms | +23.5% | | stablecoin_1-4 | 227.2 μs | 282.8 μs | +24.5% | | stablecoin_1-5 | 1.222 ms | 1.507 ms | +23.3% | | stablecoin_1-6 | 226.0 μs | 282.5 μs | +25.0% | | stablecoin_2-1 | 1.197 ms | 1.507 ms | +25.9% | | stablecoin_2-2 | 226.8 μs | 282.9 μs | +24.7% | | stablecoin_2-3 | 1.220 ms | 1.506 ms | +23.4% | | stablecoin_2-4 | 227.2 μs | 282.7 μs | +24.4% | | token-account-1 | 328.8 μs | 404.0 μs | +22.9% | | token-account-2 | 293.5 μs | 365.6 μs | +24.6% | | uniswap-1 | 333.5 μs | 408.2 μs | +22.4% | | uniswap-2 | 330.0 μs | 403.7 μs | +22.3% | | uniswap-3 | 1.026 ms | 1.259 ms | +22.7% | | uniswap-4 | 246.1 μs | 304.9 μs | +23.9% | | uniswap-5 | 1.024 ms | 1.257 ms | +22.8% | | uniswap-6 | 247.1 μs | 303.7 μs | +22.9% | | vesting-1 | 442.9 μs | 558.3 μs | +26.1% |
1d9a758 a803b77 Change
TOTAL 44.13 ms 54.90 ms +24.4%
github-actions[bot] commented 1 week ago

Click here to check the status of your benchmark.

github-actions[bot] commented 1 week ago

Comparing benchmark results of 'validation-decode' on '1d9a758c1f' (base) and 'a803b77a84' (PR)

Results table | Script | 1d9a758 | a803b77 | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 267.4 μs | 334.0 μs | +24.9% | | auction_1-2 | 757.5 μs | 959.0 μs | +26.6% | | auction_1-3 | 757.7 μs | 950.6 μs | +25.5% | | auction_1-4 | 273.8 μs | 334.8 μs | +22.3% | | auction_2-1 | 267.7 μs | 333.7 μs | +24.7% | | auction_2-2 | 758.1 μs | 962.8 μs | +27.0% | | auction_2-3 | 759.2 μs | 957.9 μs | +26.2% | | auction_2-4 | 757.1 μs | 963.8 μs | +27.3% | | auction_2-5 | 268.4 μs | 334.5 μs | +24.6% | | crowdfunding-success-1 | 325.7 μs | 409.6 μs | +25.8% | | crowdfunding-success-2 | 325.8 μs | 410.7 μs | +26.1% | | crowdfunding-success-3 | 325.6 μs | 411.0 μs | +26.2% | | currency-1 | 322.4 μs | 407.5 μs | +26.4% | | escrow-redeem_1-1 | 436.0 μs | 551.2 μs | +26.4% | | escrow-redeem_1-2 | 436.6 μs | 551.2 μs | +26.2% | | escrow-redeem_2-1 | 435.4 μs | 551.5 μs | +26.7% | | escrow-redeem_2-2 | 439.0 μs | 552.0 μs | +25.7% | | escrow-redeem_2-3 | 433.1 μs | 550.7 μs | +27.2% | | escrow-refund-1 | 436.2 μs | 551.7 μs | +26.5% | | future-increase-margin-1 | 329.9 μs | 408.4 μs | +23.8% | | future-increase-margin-2 | 440.5 μs | 557.4 μs | +26.5% | | future-increase-margin-3 | 438.2 μs | 557.3 μs | +27.2% | | future-increase-margin-4 | 927.0 μs | 1.197 ms | +29.1% | | future-increase-margin-5 | 926.8 μs | 1.194 ms | +28.8% | | future-pay-out-1 | 328.9 μs | 408.6 μs | +24.2% | | future-pay-out-2 | 439.8 μs | 558.5 μs | +27.0% | | future-pay-out-3 | 438.8 μs | 557.8 μs | +27.1% | | future-pay-out-4 | 929.9 μs | 1.193 ms | +28.3% | | future-settle-early-1 | 319.2 μs | 408.2 μs | +27.9% | | future-settle-early-2 | 438.4 μs | 557.2 μs | +27.1% | | future-settle-early-3 | 439.8 μs | 558.5 μs | +27.0% | | future-settle-early-4 | 930.5 μs | 1.196 ms | +28.5% | | game-sm-success_1-1 | 716.1 μs | 920.5 μs | +28.5% | | game-sm-success_1-2 | 225.6 μs | 285.1 μs | +26.4% | | game-sm-success_1-3 | 713.7 μs | 922.0 μs | +29.2% | | game-sm-success_1-4 | 225.7 μs | 285.4 μs | +26.5% | | game-sm-success_2-1 | 732.4 μs | 923.0 μs | +26.0% | | game-sm-success_2-2 | 226.0 μs | 285.4 μs | +26.3% | | game-sm-success_2-3 | 714.1 μs | 922.4 μs | +29.2% | | game-sm-success_2-4 | 225.5 μs | 286.3 μs | +27.0% | | game-sm-success_2-5 | 732.5 μs | 921.1 μs | +25.7% | | game-sm-success_2-6 | 225.3 μs | 286.3 μs | +27.1% | | multisig-sm-1 | 791.9 μs | 1.026 ms | +29.6% | | multisig-sm-2 | 795.5 μs | 1.025 ms | +28.8% | | multisig-sm-3 | 795.6 μs | 1.025 ms | +28.8% | | multisig-sm-4 | 795.3 μs | 1.025 ms | +28.9% | | multisig-sm-5 | 792.4 μs | 1.026 ms | +29.5% | | multisig-sm-6 | 813.8 μs | 1.025 ms | +26.0% | | multisig-sm-7 | 795.4 μs | 1.026 ms | +29.0% | | multisig-sm-8 | 794.3 μs | 1.023 ms | +28.8% | | multisig-sm-9 | 814.3 μs | 1.029 ms | +26.4% | | multisig-sm-10 | 789.2 μs | 1.025 ms | +29.9% | | ping-pong-1 | 666.6 μs | 857.0 μs | +28.6% | | ping-pong-2 | 667.1 μs | 858.6 μs | +28.7% | | ping-pong_2-1 | 666.5 μs | 857.0 μs | +28.6% | | prism-1 | 221.9 μs | 277.5 μs | +25.1% | | prism-2 | 712.8 μs | 910.3 μs | +27.7% | | prism-3 | 325.1 μs | 417.8 μs | +28.5% | | pubkey-1 | 227.3 μs | 287.1 μs | +26.3% | | stablecoin_1-1 | 1.192 ms | 1.520 ms | +27.5% | | stablecoin_1-2 | 225.5 μs | 287.0 μs | +27.3% | | stablecoin_1-3 | 1.199 ms | 1.518 ms | +26.6% | | stablecoin_1-4 | 226.0 μs | 286.5 μs | +26.8% | | stablecoin_1-5 | 1.203 ms | 1.516 ms | +26.0% | | stablecoin_1-6 | 226.3 μs | 285.7 μs | +26.2% | | stablecoin_2-1 | 1.222 ms | 1.517 ms | +24.1% | | stablecoin_2-2 | 227.0 μs | 286.0 μs | +26.0% | | stablecoin_2-3 | 1.219 ms | 1.520 ms | +24.7% | | stablecoin_2-4 | 226.7 μs | 287.1 μs | +26.6% | | token-account-1 | 323.4 μs | 411.9 μs | +27.4% | | token-account-2 | 292.9 μs | 369.9 μs | +26.3% | | uniswap-1 | 325.8 μs | 415.9 μs | +27.7% | | uniswap-2 | 329.7 μs | 407.3 μs | +23.5% | | uniswap-3 | 1.027 ms | 1.282 ms | +24.8% | | uniswap-4 | 242.6 μs | 307.7 μs | +26.8% | | uniswap-5 | 1.025 ms | 1.279 ms | +24.8% | | uniswap-6 | 242.9 μs | 309.1 μs | +27.3% | | vesting-1 | 442.7 μs | 563.7 μs | +27.3% |
1d9a758 a803b77 Change
TOTAL 43.73 ms 55.56 ms +27.0%