IntersectMBO / plutus

The Plutus language implementation and tools
Apache License 2.0
1.57k stars 479 forks source link

[Evaluation] [Performance] Ditch 'NamedDeBruijn' in favor of 'DeBruijn' #6706

Closed effectfully closed 1 day ago

effectfully commented 4 days ago

Let's see how much these nasty name conversions cost us.

effectfully commented 4 days ago

/benchmark validation

effectfully commented 4 days ago

/benchmark validation

effectfully commented 4 days ago

/benchmark validation

github-actions[bot] commented 4 days ago

Click here to check the status of your benchmark.

github-actions[bot] commented 4 days ago

Comparing benchmark results of 'validation' on '5eb80f3fe3' (base) and '4a6c11ad07' (PR)

Results table | Script | 5eb80f3 | 4a6c11a | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 241.1 μs | 250.5 μs | +3.9% | | auction_1-2 | 785.1 μs | 851.1 μs | +8.4% | | auction_1-3 | 771.8 μs | 838.3 μs | +8.6% | | auction_1-4 | 313.2 μs | 326.1 μs | +4.1% | | auction_2-1 | 241.1 μs | 251.4 μs | +4.3% | | auction_2-2 | 791.2 μs | 848.8 μs | +7.3% | | auction_2-3 | 1.023 ms | 1.092 ms | +6.7% | | auction_2-4 | 772.2 μs | 833.7 μs | +8.0% | | auction_2-5 | 324.2 μs | 326.5 μs | +0.7% | | crowdfunding-success-1 | 280.1 μs | 296.8 μs | +6.0% | | crowdfunding-success-2 | 282.0 μs | 296.2 μs | +5.0% | | crowdfunding-success-3 | 281.3 μs | 297.7 μs | +5.8% | | currency-1 | 315.9 μs | 333.5 μs | +5.6% | | escrow-redeem_1-1 | 440.0 μs | 466.9 μs | +6.1% | | escrow-redeem_1-2 | 441.9 μs | 465.6 μs | +5.4% | | escrow-redeem_2-1 | 515.2 μs | 547.1 μs | +6.2% | | escrow-redeem_2-2 | 512.4 μs | 549.4 μs | +7.2% | | escrow-redeem_2-3 | 511.5 μs | 548.9 μs | +7.3% | | escrow-refund-1 | 208.2 μs | 217.9 μs | +4.7% | | future-increase-margin-1 | 314.5 μs | 335.0 μs | +6.5% | | future-increase-margin-2 | 670.2 μs | 720.8 μs | +7.5% | | future-increase-margin-3 | 668.0 μs | 718.5 μs | +7.6% | | future-increase-margin-4 | 610.3 μs | 651.7 μs | +6.8% | | future-increase-margin-5 | 997.4 μs | 1.064 ms | +6.7% | | future-pay-out-1 | 313.2 μs | 335.2 μs | +7.0% | | future-pay-out-2 | 669.5 μs | 716.8 μs | +7.1% | | future-pay-out-3 | 676.4 μs | 719.2 μs | +6.3% | | future-pay-out-4 | 999.7 μs | 1.058 ms | +5.8% | | future-settle-early-1 | 313.9 μs | 335.2 μs | +6.8% | | future-settle-early-2 | 670.7 μs | 716.7 μs | +6.9% | | future-settle-early-3 | 673.1 μs | 724.2 μs | +7.6% | | future-settle-early-4 | 770.9 μs | 807.4 μs | +4.7% | | game-sm-success_1-1 | 488.0 μs | 515.1 μs | +5.6% | | game-sm-success_1-2 | 269.5 μs | 283.2 μs | +5.1% | | game-sm-success_1-3 | 793.3 μs | 845.4 μs | +6.6% | | game-sm-success_1-4 | 314.5 μs | 327.5 μs | +4.1% | | game-sm-success_2-1 | 489.6 μs | 517.6 μs | +5.7% | | game-sm-success_2-2 | 271.2 μs | 284.0 μs | +4.7% | | game-sm-success_2-3 | 794.3 μs | 845.4 μs | +6.4% | | game-sm-success_2-4 | 315.0 μs | 328.2 μs | +4.2% | | game-sm-success_2-5 | 795.4 μs | 845.6 μs | +6.3% | | game-sm-success_2-6 | 314.7 μs | 328.3 μs | +4.3% | | multisig-sm-1 | 498.1 μs | 523.8 μs | +5.2% | | multisig-sm-2 | 488.2 μs | 511.5 μs | +4.8% | | multisig-sm-3 | 485.8 μs | 517.9 μs | +6.6% | | multisig-sm-4 | 498.5 μs | 525.5 μs | +5.4% | | multisig-sm-5 | 684.2 μs | 733.3 μs | +7.2% | | multisig-sm-6 | 500.1 μs | 525.1 μs | +5.0% | | multisig-sm-7 | 486.2 μs | 512.3 μs | +5.4% | | multisig-sm-8 | 485.6 μs | 518.6 μs | +6.8% | | multisig-sm-9 | 500.4 μs | 528.9 μs | +5.7% | | multisig-sm-10 | 692.8 μs | 736.9 μs | +6.4% | | ping-pong-1 | 411.4 μs | 424.7 μs | +3.2% | | ping-pong-2 | 412.8 μs | 424.0 μs | +2.7% | | ping-pong_2-1 | 260.3 μs | 272.9 μs | +4.8% | | prism-1 | 229.0 μs | 237.7 μs | +3.8% | | prism-2 | 530.3 μs | 557.2 μs | +5.1% | | prism-3 | 472.4 μs | 496.6 μs | +5.1% | | pubkey-1 | 191.9 μs | 202.9 μs | +5.7% | | stablecoin_1-1 | 1.163 ms | 1.197 ms | +2.9% | | stablecoin_1-2 | 264.2 μs | 279.0 μs | +5.6% | | stablecoin_1-3 | 1.314 ms | 1.380 ms | +5.0% | | stablecoin_1-4 | 281.0 μs | 294.8 μs | +4.9% | | stablecoin_1-5 | 1.682 ms | 1.763 ms | +4.8% | | stablecoin_1-6 | 347.0 μs | 363.7 μs | +4.8% | | stablecoin_2-1 | 1.164 ms | 1.201 ms | +3.2% | | stablecoin_2-2 | 264.7 μs | 277.4 μs | +4.8% | | stablecoin_2-3 | 1.325 ms | 1.380 ms | +4.2% | | stablecoin_2-4 | 281.3 μs | 293.6 μs | +4.4% | | token-account-1 | 243.7 μs | 255.2 μs | +4.7% | | token-account-2 | 427.4 μs | 452.3 μs | +5.8% | | uniswap-1 | 500.7 μs | 532.2 μs | +6.3% | | uniswap-2 | 286.2 μs | 304.9 μs | +6.5% | | uniswap-3 | 2.143 ms | 2.282 ms | +6.5% | | uniswap-4 | 449.4 μs | 464.7 μs | +3.4% | | uniswap-5 | 1.439 ms | 1.524 ms | +5.9% | | uniswap-6 | 426.7 μs | 445.2 μs | +4.3% | | vesting-1 | 437.9 μs | 463.4 μs | +5.8% |
5eb80f3 4a6c11a Change
TOTAL 44.51 ms 47.06 ms +5.7%
github-actions[bot] commented 4 days ago

Click here to check the status of your benchmark.

effectfully commented 4 days ago

WWWWAT hahaha

github-actions[bot] commented 4 days ago

Comparing benchmark results of 'validation' on '5eb80f3fe3' (base) and '4a6c11ad07' (PR)

Results table | Script | 5eb80f3 | 4a6c11a | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 239.4 μs | 253.7 μs | +6.0% | | auction_1-2 | 782.0 μs | 851.8 μs | +8.9% | | auction_1-3 | 768.6 μs | 838.4 μs | +9.1% | | auction_1-4 | 310.2 μs | 329.4 μs | +6.2% | | auction_2-1 | 241.8 μs | 253.8 μs | +5.0% | | auction_2-2 | 781.6 μs | 849.6 μs | +8.7% | | auction_2-3 | 1.019 ms | 1.095 ms | +7.5% | | auction_2-4 | 767.6 μs | 838.3 μs | +9.2% | | auction_2-5 | 321.1 μs | 329.1 μs | +2.5% | | crowdfunding-success-1 | 279.1 μs | 302.3 μs | +8.3% | | crowdfunding-success-2 | 279.6 μs | 301.6 μs | +7.9% | | crowdfunding-success-3 | 279.5 μs | 300.0 μs | +7.3% | | currency-1 | 319.2 μs | 336.1 μs | +5.3% | | escrow-redeem_1-1 | 438.0 μs | 468.7 μs | +7.0% | | escrow-redeem_1-2 | 437.1 μs | 470.1 μs | +7.5% | | escrow-redeem_2-1 | 508.7 μs | 550.8 μs | +8.3% | | escrow-redeem_2-2 | 511.6 μs | 553.4 μs | +8.2% | | escrow-redeem_2-3 | 509.2 μs | 551.3 μs | +8.3% | | escrow-refund-1 | 205.8 μs | 220.4 μs | +7.1% | | future-increase-margin-1 | 311.7 μs | 337.0 μs | +8.1% | | future-increase-margin-2 | 666.5 μs | 722.1 μs | +8.3% | | future-increase-margin-3 | 666.6 μs | 721.8 μs | +8.3% | | future-increase-margin-4 | 603.2 μs | 650.8 μs | +7.9% | | future-increase-margin-5 | 992.4 μs | 1.069 ms | +7.7% | | future-pay-out-1 | 311.2 μs | 336.0 μs | +8.0% | | future-pay-out-2 | 667.4 μs | 720.4 μs | +7.9% | | future-pay-out-3 | 664.1 μs | 723.7 μs | +9.0% | | future-pay-out-4 | 1.001 ms | 1.063 ms | +6.2% | | future-settle-early-1 | 313.6 μs | 336.7 μs | +7.4% | | future-settle-early-2 | 664.0 μs | 720.5 μs | +8.5% | | future-settle-early-3 | 670.2 μs | 722.5 μs | +7.8% | | future-settle-early-4 | 761.6 μs | 806.0 μs | +5.8% | | game-sm-success_1-1 | 485.1 μs | 518.4 μs | +6.9% | | game-sm-success_1-2 | 267.2 μs | 286.2 μs | +7.1% | | game-sm-success_1-3 | 789.2 μs | 847.7 μs | +7.4% | | game-sm-success_1-4 | 312.5 μs | 331.2 μs | +6.0% | | game-sm-success_2-1 | 487.3 μs | 519.3 μs | +6.6% | | game-sm-success_2-2 | 267.6 μs | 286.0 μs | +6.9% | | game-sm-success_2-3 | 790.1 μs | 846.4 μs | +7.1% | | game-sm-success_2-4 | 312.6 μs | 331.9 μs | +6.2% | | game-sm-success_2-5 | 798.3 μs | 847.2 μs | +6.1% | | game-sm-success_2-6 | 312.1 μs | 332.9 μs | +6.7% | | multisig-sm-1 | 503.7 μs | 526.0 μs | +4.4% | | multisig-sm-2 | 495.4 μs | 515.3 μs | +4.0% | | multisig-sm-3 | 485.2 μs | 520.2 μs | +7.2% | | multisig-sm-4 | 493.0 μs | 526.1 μs | +6.7% | | multisig-sm-5 | 679.6 μs | 737.0 μs | +8.4% | | multisig-sm-6 | 495.6 μs | 525.6 μs | +6.1% | | multisig-sm-7 | 486.1 μs | 514.7 μs | +5.9% | | multisig-sm-8 | 482.3 μs | 520.0 μs | +7.8% | | multisig-sm-9 | 497.4 μs | 531.1 μs | +6.8% | | multisig-sm-10 | 689.4 μs | 737.2 μs | +6.9% | | ping-pong-1 | 409.5 μs | 426.1 μs | +4.1% | | ping-pong-2 | 417.0 μs | 424.5 μs | +1.8% | | ping-pong_2-1 | 256.1 μs | 276.8 μs | +8.1% | | prism-1 | 227.2 μs | 240.0 μs | +5.6% | | prism-2 | 523.4 μs | 559.9 μs | +7.0% | | prism-3 | 469.0 μs | 498.8 μs | +6.4% | | pubkey-1 | 189.6 μs | 204.3 μs | +7.8% | | stablecoin_1-1 | 1.164 ms | 1.197 ms | +2.8% | | stablecoin_1-2 | 261.5 μs | 280.9 μs | +7.4% | | stablecoin_1-3 | 1.328 ms | 1.379 ms | +3.8% | | stablecoin_1-4 | 278.6 μs | 297.2 μs | +6.7% | | stablecoin_1-5 | 1.685 ms | 1.762 ms | +4.6% | | stablecoin_1-6 | 351.6 μs | 366.3 μs | +4.2% | | stablecoin_2-1 | 1.157 ms | 1.201 ms | +3.8% | | stablecoin_2-2 | 261.8 μs | 281.3 μs | +7.4% | | stablecoin_2-3 | 1.321 ms | 1.381 ms | +4.5% | | stablecoin_2-4 | 280.1 μs | 296.5 μs | +5.9% | | token-account-1 | 241.9 μs | 256.5 μs | +6.0% | | token-account-2 | 424.5 μs | 455.1 μs | +7.2% | | uniswap-1 | 498.5 μs | 533.1 μs | +6.9% | | uniswap-2 | 283.9 μs | 307.1 μs | +8.2% | | uniswap-3 | 2.131 ms | 2.276 ms | +6.8% | | uniswap-4 | 445.6 μs | 470.4 μs | +5.6% | | uniswap-5 | 1.431 ms | 1.527 ms | +6.7% | | uniswap-6 | 424.0 μs | 450.3 μs | +6.2% | | vesting-1 | 438.0 μs | 465.0 μs | +6.2% |
5eb80f3 4a6c11a Change
TOTAL 44.32 ms 47.24 ms +6.6%
github-actions[bot] commented 4 days ago

Click here to check the status of your benchmark.

github-actions[bot] commented 4 days ago

Comparing benchmark results of 'validation' on '5eb80f3fe3' (base) and '4a6c11ad07' (PR)

Results table | Script | 5eb80f3 | 4a6c11a | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 241.6 μs | 251.5 μs | +4.1% | | auction_1-2 | 790.7 μs | 849.3 μs | +7.4% | | auction_1-3 | 771.3 μs | 836.2 μs | +8.4% | | auction_1-4 | 313.1 μs | 326.8 μs | +4.4% | | auction_2-1 | 248.3 μs | 252.1 μs | +1.5% | | auction_2-2 | 784.5 μs | 844.4 μs | +7.6% | | auction_2-3 | 1.027 ms | 1.100 ms | +7.1% | | auction_2-4 | 779.2 μs | 837.9 μs | +7.5% | | auction_2-5 | 312.8 μs | 326.8 μs | +4.5% | | crowdfunding-success-1 | 280.5 μs | 297.0 μs | +5.9% | | crowdfunding-success-2 | 281.6 μs | 298.3 μs | +5.9% | | crowdfunding-success-3 | 281.5 μs | 297.1 μs | +5.5% | | currency-1 | 321.5 μs | 334.3 μs | +4.0% | | escrow-redeem_1-1 | 440.6 μs | 465.9 μs | +5.7% | | escrow-redeem_1-2 | 440.8 μs | 467.6 μs | +6.1% | | escrow-redeem_2-1 | 515.6 μs | 546.4 μs | +6.0% | | escrow-redeem_2-2 | 512.5 μs | 547.9 μs | +6.9% | | escrow-redeem_2-3 | 517.4 μs | 546.6 μs | +5.6% | | escrow-refund-1 | 207.5 μs | 218.2 μs | +5.2% | | future-increase-margin-1 | 313.0 μs | 334.5 μs | +6.9% | | future-increase-margin-2 | 671.2 μs | 721.2 μs | +7.4% | | future-increase-margin-3 | 668.2 μs | 718.3 μs | +7.5% | | future-increase-margin-4 | 601.9 μs | 653.0 μs | +8.5% | | future-increase-margin-5 | 994.3 μs | 1.070 ms | +7.6% | | future-pay-out-1 | 314.4 μs | 335.7 μs | +6.8% | | future-pay-out-2 | 667.3 μs | 718.7 μs | +7.7% | | future-pay-out-3 | 666.8 μs | 720.7 μs | +8.1% | | future-pay-out-4 | 1.005 ms | 1.061 ms | +5.6% | | future-settle-early-1 | 315.4 μs | 336.1 μs | +6.6% | | future-settle-early-2 | 669.4 μs | 716.2 μs | +7.0% | | future-settle-early-3 | 668.7 μs | 719.6 μs | +7.6% | | future-settle-early-4 | 765.4 μs | 809.0 μs | +5.7% | | game-sm-success_1-1 | 489.1 μs | 515.9 μs | +5.5% | | game-sm-success_1-2 | 269.9 μs | 283.7 μs | +5.1% | | game-sm-success_1-3 | 793.4 μs | 845.0 μs | +6.5% | | game-sm-success_1-4 | 314.9 μs | 328.7 μs | +4.4% | | game-sm-success_2-1 | 495.1 μs | 515.3 μs | +4.1% | | game-sm-success_2-2 | 270.1 μs | 284.1 μs | +5.2% | | game-sm-success_2-3 | 792.9 μs | 844.4 μs | +6.5% | | game-sm-success_2-4 | 314.1 μs | 328.6 μs | +4.6% | | game-sm-success_2-5 | 788.0 μs | 843.9 μs | +7.1% | | game-sm-success_2-6 | 314.4 μs | 328.8 μs | +4.6% | | multisig-sm-1 | 510.2 μs | 525.9 μs | +3.1% | | multisig-sm-2 | 496.3 μs | 513.1 μs | +3.4% | | multisig-sm-3 | 485.6 μs | 519.2 μs | +6.9% | | multisig-sm-4 | 496.1 μs | 526.5 μs | +6.1% | | multisig-sm-5 | 687.2 μs | 736.4 μs | +7.2% | | multisig-sm-6 | 498.3 μs | 524.1 μs | +5.2% | | multisig-sm-7 | 498.0 μs | 513.5 μs | +3.1% | | multisig-sm-8 | 484.4 μs | 520.1 μs | +7.4% | | multisig-sm-9 | 496.2 μs | 529.4 μs | +6.7% | | multisig-sm-10 | 683.8 μs | 737.5 μs | +7.9% | | ping-pong-1 | 411.7 μs | 423.5 μs | +2.9% | | ping-pong-2 | 411.0 μs | 423.7 μs | +3.1% | | ping-pong_2-1 | 258.5 μs | 271.9 μs | +5.2% | | prism-1 | 228.9 μs | 238.8 μs | +4.3% | | prism-2 | 525.9 μs | 557.5 μs | +6.0% | | prism-3 | 472.3 μs | 496.7 μs | +5.2% | | pubkey-1 | 191.3 μs | 203.1 μs | +6.2% | | stablecoin_1-1 | 1.163 ms | 1.197 ms | +2.9% | | stablecoin_1-2 | 263.3 μs | 279.1 μs | +6.0% | | stablecoin_1-3 | 1.337 ms | 1.379 ms | +3.1% | | stablecoin_1-4 | 282.0 μs | 294.7 μs | +4.5% | | stablecoin_1-5 | 1.720 ms | 1.760 ms | +2.3% | | stablecoin_1-6 | 345.3 μs | 363.3 μs | +5.2% | | stablecoin_2-1 | 1.162 ms | 1.199 ms | +3.2% | | stablecoin_2-2 | 263.1 μs | 278.9 μs | +6.0% | | stablecoin_2-3 | 1.342 ms | 1.380 ms | +2.8% | | stablecoin_2-4 | 280.6 μs | 294.6 μs | +5.0% | | token-account-1 | 242.2 μs | 255.9 μs | +5.7% | | token-account-2 | 427.1 μs | 452.6 μs | +6.0% | | uniswap-1 | 495.1 μs | 533.4 μs | +7.7% | | uniswap-2 | 287.1 μs | 304.9 μs | +6.2% | | uniswap-3 | 2.134 ms | 2.283 ms | +7.0% | | uniswap-4 | 449.4 μs | 465.4 μs | +3.6% | | uniswap-5 | 1.467 ms | 1.524 ms | +3.9% | | uniswap-6 | 427.0 μs | 446.5 μs | +4.6% | | vesting-1 | 440.9 μs | 463.2 μs | +5.1% |
5eb80f3 4a6c11a Change
TOTAL 44.59 ms 47.09 ms +5.6%
kwxm commented 4 days ago

Those results look pretty conclusive!

zeme-wana commented 4 days ago

/benchmark validation

github-actions[bot] commented 4 days ago

Click here to check the status of your benchmark.

github-actions[bot] commented 4 days ago

Comparing benchmark results of 'validation' on '5eb80f3fe' (base) and '4a6c11ad0' (PR)

Results table | Script | 5eb80f3 | 4a6c11a | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 241.0 μs | 250.6 μs | +4.0% | | auction_1-2 | 788.9 μs | 844.1 μs | +7.0% | | auction_1-3 | 775.7 μs | 834.1 μs | +7.5% | | auction_1-4 | 312.4 μs | 325.3 μs | +4.1% | | auction_2-1 | 241.0 μs | 251.1 μs | +4.2% | | auction_2-2 | 800.3 μs | 848.9 μs | +6.1% | | auction_2-3 | 1.014 ms | 1.090 ms | +7.5% | | auction_2-4 | 773.7 μs | 835.1 μs | +7.9% | | auction_2-5 | 312.7 μs | 326.0 μs | +4.3% | | crowdfunding-success-1 | 280.9 μs | 296.9 μs | +5.7% | | crowdfunding-success-2 | 282.0 μs | 296.9 μs | +5.3% | | crowdfunding-success-3 | 281.7 μs | 297.8 μs | +5.7% | | currency-1 | 313.4 μs | 335.1 μs | +6.9% | | escrow-redeem_1-1 | 441.0 μs | 464.3 μs | +5.3% | | escrow-redeem_1-2 | 440.9 μs | 467.2 μs | +6.0% | | escrow-redeem_2-1 | 511.5 μs | 546.2 μs | +6.8% | | escrow-redeem_2-2 | 511.0 μs | 548.8 μs | +7.4% | | escrow-redeem_2-3 | 510.8 μs | 549.8 μs | +7.6% | | escrow-refund-1 | 207.6 μs | 217.7 μs | +4.9% | | future-increase-margin-1 | 316.2 μs | 333.5 μs | +5.5% | | future-increase-margin-2 | 668.2 μs | 715.8 μs | +7.1% | | future-increase-margin-3 | 667.9 μs | 717.6 μs | +7.4% | | future-increase-margin-4 | 602.8 μs | 652.8 μs | +8.3% | | future-increase-margin-5 | 995.7 μs | 1.061 ms | +6.6% | | future-pay-out-1 | 312.7 μs | 334.7 μs | +7.0% | | future-pay-out-2 | 668.7 μs | 717.4 μs | +7.3% | | future-pay-out-3 | 677.8 μs | 718.8 μs | +6.0% | | future-pay-out-4 | 995.7 μs | 1.058 ms | +6.3% | | future-settle-early-1 | 315.3 μs | 334.6 μs | +6.1% | | future-settle-early-2 | 669.3 μs | 719.3 μs | +7.5% | | future-settle-early-3 | 668.6 μs | 719.0 μs | +7.5% | | future-settle-early-4 | 755.6 μs | 806.7 μs | +6.8% | | game-sm-success_1-1 | 488.7 μs | 515.3 μs | +5.4% | | game-sm-success_1-2 | 270.2 μs | 284.1 μs | +5.1% | | game-sm-success_1-3 | 793.6 μs | 842.4 μs | +6.1% | | game-sm-success_1-4 | 314.7 μs | 327.9 μs | +4.2% | | game-sm-success_2-1 | 491.7 μs | 515.4 μs | +4.8% | | game-sm-success_2-2 | 269.9 μs | 282.6 μs | +4.7% | | game-sm-success_2-3 | 793.8 μs | 844.3 μs | +6.4% | | game-sm-success_2-4 | 314.1 μs | 329.0 μs | +4.7% | | game-sm-success_2-5 | 792.8 μs | 847.7 μs | +6.9% | | game-sm-success_2-6 | 315.0 μs | 328.1 μs | +4.2% | | multisig-sm-1 | 500.0 μs | 523.7 μs | +4.7% | | multisig-sm-2 | 485.2 μs | 511.8 μs | +5.5% | | multisig-sm-3 | 486.2 μs | 518.2 μs | +6.6% | | multisig-sm-4 | 500.4 μs | 524.4 μs | +4.8% | | multisig-sm-5 | 690.6 μs | 733.3 μs | +6.2% | | multisig-sm-6 | 500.3 μs | 523.1 μs | +4.6% | | multisig-sm-7 | 486.1 μs | 511.6 μs | +5.2% | | multisig-sm-8 | 483.2 μs | 516.6 μs | +6.9% | | multisig-sm-9 | 494.9 μs | 528.6 μs | +6.8% | | multisig-sm-10 | 683.4 μs | 734.5 μs | +7.5% | | ping-pong-1 | 412.2 μs | 425.6 μs | +3.3% | | ping-pong-2 | 412.5 μs | 423.2 μs | +2.6% | | ping-pong_2-1 | 260.2 μs | 271.5 μs | +4.3% | | prism-1 | 229.0 μs | 237.1 μs | +3.5% | | prism-2 | 524.5 μs | 554.2 μs | +5.7% | | prism-3 | 484.0 μs | 496.0 μs | +2.5% | | pubkey-1 | 192.5 μs | 201.9 μs | +4.9% | | stablecoin_1-1 | 1.146 ms | 1.198 ms | +4.5% | | stablecoin_1-2 | 263.9 μs | 277.5 μs | +5.2% | | stablecoin_1-3 | 1.323 ms | 1.381 ms | +4.4% | | stablecoin_1-4 | 281.5 μs | 293.4 μs | +4.2% | | stablecoin_1-5 | 1.682 ms | 1.762 ms | +4.8% | | stablecoin_1-6 | 346.0 μs | 361.5 μs | +4.5% | | stablecoin_2-1 | 1.154 ms | 1.200 ms | +4.0% | | stablecoin_2-2 | 271.0 μs | 277.5 μs | +2.4% | | stablecoin_2-3 | 1.337 ms | 1.380 ms | +3.2% | | stablecoin_2-4 | 281.0 μs | 293.1 μs | +4.3% | | token-account-1 | 243.0 μs | 255.6 μs | +5.2% | | token-account-2 | 427.8 μs | 451.9 μs | +5.6% | | uniswap-1 | 498.5 μs | 533.3 μs | +7.0% | | uniswap-2 | 286.2 μs | 302.5 μs | +5.7% | | uniswap-3 | 2.139 ms | 2.283 ms | +6.7% | | uniswap-4 | 448.4 μs | 464.2 μs | +3.5% | | uniswap-5 | 1.441 ms | 1.526 ms | +5.9% | | uniswap-6 | 426.8 μs | 445.0 μs | +4.3% | | vesting-1 | 437.7 μs | 462.8 μs | +5.7% |
5eb80f3 4a6c11a Change
TOTAL 44.46 ms 47.02 ms +5.8%
effectfully commented 4 days ago

OK, I'm a moron, validation benchmarks don't include the preparation costs. It's still wild that we get a slowdown here, as if toNamedDeBruijn was forcing something important that somehow isn't forced by force. Weird.

effectfully commented 4 days ago

/benchmark validation-full

effectfully commented 4 days ago

/benchmark validation-full

effectfully commented 4 days ago

/benchmark validation-full

github-actions[bot] commented 4 days ago

Click here to check the status of your benchmark.

github-actions[bot] commented 4 days ago

Comparing benchmark results of 'validation-full' on '5eb80f3fe' (base) and '4a6c11ad0' (PR)

Results table | Script | 5eb80f3 | 4a6c11a | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 601.5 μs | 613.9 μs | +2.1% | | auction_1-2 | 1.976 ms | 2.013 ms | +1.9% | | auction_1-3 | 1.953 ms | 2.011 ms | +3.0% | | auction_1-4 | 680.3 μs | 697.9 μs | +2.6% | | auction_2-1 | 601.7 μs | 612.9 μs | +1.9% | | auction_2-2 | 1.958 ms | 2.013 ms | +2.8% | | auction_2-3 | 2.270 ms | 2.336 ms | +2.9% | | auction_2-4 | 1.961 ms | 2.012 ms | +2.6% | | auction_2-5 | 679.8 μs | 697.2 μs | +2.6% | | crowdfunding-success-1 | 730.3 μs | 743.0 μs | +1.7% | | crowdfunding-success-2 | 731.3 μs | 741.8 μs | +1.4% | | crowdfunding-success-3 | 730.7 μs | 743.0 μs | +1.7% | | currency-1 | 778.8 μs | 795.3 μs | +2.1% | | escrow-redeem_1-1 | 1.079 ms | 1.106 ms | +2.5% | | escrow-redeem_1-2 | 1.077 ms | 1.105 ms | +2.6% | | escrow-redeem_2-1 | 1.162 ms | 1.195 ms | +2.8% | | escrow-redeem_2-2 | 1.161 ms | 1.206 ms | +3.9% | | escrow-redeem_2-3 | 1.162 ms | 1.195 ms | +2.8% | | escrow-refund-1 | 797.4 μs | 804.0 μs | +0.8% | | future-increase-margin-1 | 781.2 μs | 794.1 μs | +1.7% | | future-increase-margin-2 | 1.349 ms | 1.200 ms | -11.0% | | future-increase-margin-3 | 1.350 ms | 1.198 ms | -11.3% | | future-increase-margin-4 | 2.000 ms | 2.044 ms | +2.2% | | future-increase-margin-5 | 2.495 ms | 2.611 ms | +4.6% | | future-pay-out-1 | 780.8 μs | 794.7 μs | +1.8% | | future-pay-out-2 | 1.350 ms | 1.200 ms | -11.1% | | future-pay-out-3 | 1.349 ms | 1.200 ms | -11.0% | | future-pay-out-4 | 2.493 ms | 2.571 ms | +3.1% | | future-settle-early-1 | 774.1 μs | 794.9 μs | +2.7% | | future-settle-early-2 | 1.351 ms | 1.200 ms | -11.2% | | future-settle-early-3 | 1.350 ms | 1.198 ms | -11.3% | | future-settle-early-4 | 2.192 ms | 2.248 ms | +2.6% | | game-sm-success_1-1 | 1.551 ms | 1.570 ms | +1.2% | | game-sm-success_1-2 | 578.0 μs | 598.0 μs | +3.5% | | game-sm-success_1-3 | 1.943 ms | 1.998 ms | +2.8% | | game-sm-success_1-4 | 624.1 μs | 649.1 μs | +4.0% | | game-sm-success_2-1 | 1.561 ms | 1.570 ms | +0.6% | | game-sm-success_2-2 | 578.5 μs | 598.8 μs | +3.5% | | game-sm-success_2-3 | 1.929 ms | 1.992 ms | +3.3% | | game-sm-success_2-4 | 624.4 μs | 649.5 μs | +4.0% | | game-sm-success_2-5 | 1.941 ms | 2.000 ms | +3.0% | | game-sm-success_2-6 | 624.8 μs | 649.8 μs | +4.0% | | multisig-sm-1 | 1.691 ms | 1.741 ms | +3.0% | | multisig-sm-2 | 1.660 ms | 1.685 ms | +1.5% | | multisig-sm-3 | 1.678 ms | 1.699 ms | +1.3% | | multisig-sm-4 | 1.670 ms | 1.699 ms | +1.7% | | multisig-sm-5 | 1.937 ms | 1.974 ms | +1.9% | | multisig-sm-6 | 1.675 ms | 1.700 ms | +1.5% | | multisig-sm-7 | 1.660 ms | 1.681 ms | +1.3% | | multisig-sm-8 | 1.677 ms | 1.716 ms | +2.3% | | multisig-sm-9 | 1.686 ms | 1.697 ms | +0.7% | | multisig-sm-10 | 1.916 ms | 1.971 ms | +2.9% | | ping-pong-1 | 1.387 ms | 1.408 ms | +1.5% | | ping-pong-2 | 1.399 ms | 1.424 ms | +1.8% | | ping-pong_2-1 | 1.193 ms | 1.251 ms | +4.9% | | prism-1 | 518.6 μs | 539.6 μs | +4.0% | | prism-2 | 1.569 ms | 1.602 ms | +2.1% | | prism-3 | 949.4 μs | 981.2 μs | +3.3% | | pubkey-1 | 493.8 μs | 502.5 μs | +1.8% | | stablecoin_1-1 | 3.093 ms | 3.135 ms | +1.4% | | stablecoin_1-2 | 568.2 μs | 590.9 μs | +4.0% | | stablecoin_1-3 | 3.336 ms | 3.373 ms | +1.1% | | stablecoin_1-4 | 587.7 μs | 609.3 μs | +3.7% | | stablecoin_1-5 | 3.799 ms | 3.927 ms | +3.4% | | stablecoin_1-6 | 660.5 μs | 681.2 μs | +3.1% | | stablecoin_2-1 | 3.107 ms | 3.125 ms | +0.6% | | stablecoin_2-2 | 568.6 μs | 590.8 μs | +3.9% | | stablecoin_2-3 | 3.323 ms | 3.372 ms | +1.5% | | stablecoin_2-4 | 587.5 μs | 609.2 μs | +3.7% | | token-account-1 | 691.8 μs | 700.0 μs | +1.2% | | token-account-2 | 857.9 μs | 897.1 μs | +4.6% | | uniswap-1 | 969.3 μs | 1.033 ms | +6.6% | | uniswap-2 | 736.7 μs | 755.8 μs | +2.6% | | uniswap-3 | 4.033 ms | 4.243 ms | +5.2% | | uniswap-4 | 795.9 μs | 822.4 μs | +3.3% | | uniswap-5 | 3.070 ms | 3.187 ms | +3.8% | | uniswap-6 | 775.7 μs | 799.0 μs | +3.0% | | vesting-1 | 1.089 ms | 1.120 ms | +2.8% |
5eb80f3 4a6c11a Change
TOTAL 111.1 ms 112.8 ms +1.6%
github-actions[bot] commented 4 days ago

Click here to check the status of your benchmark.

effectfully commented 4 days ago

WTF is going on in this PR

image

github-actions[bot] commented 4 days ago

Comparing benchmark results of 'validation-full' on '5eb80f3fe' (base) and '4a6c11ad0' (PR)

Results table | Script | 5eb80f3 | 4a6c11a | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 603.3 μs | 613.1 μs | +1.6% | | auction_1-2 | 1.977 ms | 2.014 ms | +1.9% | | auction_1-3 | 1.968 ms | 2.014 ms | +2.3% | | auction_1-4 | 680.8 μs | 697.8 μs | +2.5% | | auction_2-1 | 603.8 μs | 613.1 μs | +1.5% | | auction_2-2 | 1.959 ms | 2.014 ms | +2.8% | | auction_2-3 | 2.273 ms | 2.338 ms | +2.9% | | auction_2-4 | 1.968 ms | 2.015 ms | +2.4% | | auction_2-5 | 680.4 μs | 697.7 μs | +2.5% | | crowdfunding-success-1 | 733.0 μs | 743.8 μs | +1.5% | | crowdfunding-success-2 | 731.8 μs | 743.1 μs | +1.5% | | crowdfunding-success-3 | 732.4 μs | 743.5 μs | +1.5% | | currency-1 | 778.6 μs | 796.2 μs | +2.3% | | escrow-redeem_1-1 | 1.077 ms | 1.108 ms | +2.9% | | escrow-redeem_1-2 | 1.079 ms | 1.108 ms | +2.7% | | escrow-redeem_2-1 | 1.164 ms | 1.197 ms | +2.8% | | escrow-redeem_2-2 | 1.163 ms | 1.197 ms | +2.9% | | escrow-redeem_2-3 | 1.162 ms | 1.196 ms | +2.9% | | escrow-refund-1 | 797.9 μs | 809.3 μs | +1.4% | | future-increase-margin-1 | 775.0 μs | 794.7 μs | +2.5% | | future-increase-margin-2 | 1.352 ms | 1.201 ms | -11.2% | | future-increase-margin-3 | 1.348 ms | 1.201 ms | -10.9% | | future-increase-margin-4 | 2.002 ms | 2.043 ms | +2.0% | | future-increase-margin-5 | 2.497 ms | 2.586 ms | +3.6% | | future-pay-out-1 | 776.7 μs | 794.8 μs | +2.3% | | future-pay-out-2 | 1.349 ms | 1.201 ms | -11.0% | | future-pay-out-3 | 1.349 ms | 1.199 ms | -11.1% | | future-pay-out-4 | 2.492 ms | 2.572 ms | +3.2% | | future-settle-early-1 | 782.2 μs | 795.0 μs | +1.6% | | future-settle-early-2 | 1.350 ms | 1.201 ms | -11.0% | | future-settle-early-3 | 1.351 ms | 1.199 ms | -11.3% | | future-settle-early-4 | 2.180 ms | 2.267 ms | +4.0% | | game-sm-success_1-1 | 1.546 ms | 1.572 ms | +1.7% | | game-sm-success_1-2 | 578.8 μs | 598.8 μs | +3.5% | | game-sm-success_1-3 | 1.943 ms | 1.993 ms | +2.6% | | game-sm-success_1-4 | 625.5 μs | 649.0 μs | +3.8% | | game-sm-success_2-1 | 1.559 ms | 1.571 ms | +0.8% | | game-sm-success_2-2 | 578.8 μs | 598.2 μs | +3.4% | | game-sm-success_2-3 | 1.943 ms | 1.992 ms | +2.5% | | game-sm-success_2-4 | 625.5 μs | 648.9 μs | +3.7% | | game-sm-success_2-5 | 1.927 ms | 1.992 ms | +3.4% | | game-sm-success_2-6 | 624.5 μs | 650.4 μs | +4.1% | | multisig-sm-1 | 1.675 ms | 1.705 ms | +1.8% | | multisig-sm-2 | 1.657 ms | 1.682 ms | +1.5% | | multisig-sm-3 | 1.666 ms | 1.693 ms | +1.6% | | multisig-sm-4 | 1.669 ms | 1.700 ms | +1.9% | | multisig-sm-5 | 1.920 ms | 1.985 ms | +3.4% | | multisig-sm-6 | 1.696 ms | 1.703 ms | +0.4% | | multisig-sm-7 | 1.654 ms | 1.681 ms | +1.6% | | multisig-sm-8 | 1.661 ms | 1.714 ms | +3.2% | | multisig-sm-9 | 1.671 ms | 1.702 ms | +1.9% | | multisig-sm-10 | 1.917 ms | 1.971 ms | +2.8% | | ping-pong-1 | 1.381 ms | 1.405 ms | +1.7% | | ping-pong-2 | 1.380 ms | 1.412 ms | +2.3% | | ping-pong_2-1 | 1.191 ms | 1.209 ms | +1.5% | | prism-1 | 518.0 μs | 539.6 μs | +4.2% | | prism-2 | 1.570 ms | 1.602 ms | +2.0% | | prism-3 | 955.0 μs | 983.2 μs | +3.0% | | pubkey-1 | 493.7 μs | 503.6 μs | +2.0% | | stablecoin_1-1 | 3.093 ms | 3.126 ms | +1.1% | | stablecoin_1-2 | 568.2 μs | 591.3 μs | +4.1% | | stablecoin_1-3 | 3.325 ms | 3.375 ms | +1.5% | | stablecoin_1-4 | 587.5 μs | 610.1 μs | +3.8% | | stablecoin_1-5 | 3.797 ms | 3.935 ms | +3.6% | | stablecoin_1-6 | 661.2 μs | 681.8 μs | +3.1% | | stablecoin_2-1 | 3.095 ms | 3.125 ms | +1.0% | | stablecoin_2-2 | 568.4 μs | 592.3 μs | +4.2% | | stablecoin_2-3 | 3.324 ms | 3.376 ms | +1.6% | | stablecoin_2-4 | 588.0 μs | 609.9 μs | +3.7% | | token-account-1 | 692.4 μs | 700.3 μs | +1.1% | | token-account-2 | 858.3 μs | 897.3 μs | +4.5% | | uniswap-1 | 975.1 μs | 1.035 ms | +6.1% | | uniswap-2 | 743.8 μs | 755.7 μs | +1.6% | | uniswap-3 | 4.035 ms | 4.255 ms | +5.5% | | uniswap-4 | 797.4 μs | 822.6 μs | +3.2% | | uniswap-5 | 3.071 ms | 3.193 ms | +4.0% | | uniswap-6 | 778.2 μs | 799.5 μs | +2.7% | | vesting-1 | 1.092 ms | 1.120 ms | +2.6% |
5eb80f3 4a6c11a Change
TOTAL 111.0 ms 112.8 ms +1.6%
github-actions[bot] commented 4 days ago

Click here to check the status of your benchmark.

github-actions[bot] commented 4 days ago

Comparing benchmark results of 'validation-full' on '5eb80f3fe' (base) and '4a6c11ad0' (PR)

Results table | Script | 5eb80f3 | 4a6c11a | Change | | :------ | :------: | :------: | :------: | | auction_1-1 | 611.6 μs | 605.3 μs | -1.0% | | auction_1-2 | 1.986 ms | 1.993 ms | +0.4% | | auction_1-3 | 1.974 ms | 1.992 ms | +0.9% | | auction_1-4 | 691.8 μs | 689.9 μs | -0.3% | | auction_2-1 | 612.3 μs | 605.7 μs | -1.1% | | auction_2-2 | 1.982 ms | 1.993 ms | +0.6% | | auction_2-3 | 2.280 ms | 2.315 ms | +1.5% | | auction_2-4 | 1.976 ms | 1.996 ms | +1.0% | | auction_2-5 | 690.3 μs | 689.4 μs | -0.1% | | crowdfunding-success-1 | 742.9 μs | 733.5 μs | -1.3% | | crowdfunding-success-2 | 743.7 μs | 733.1 μs | -1.4% | | crowdfunding-success-3 | 743.0 μs | 733.1 μs | -1.3% | | currency-1 | 784.1 μs | 786.9 μs | +0.4% | | escrow-redeem_1-1 | 1.094 ms | 1.094 ms | 0.0% | | escrow-redeem_1-2 | 1.093 ms | 1.094 ms | +0.1% | | escrow-redeem_2-1 | 1.179 ms | 1.183 ms | +0.3% | | escrow-redeem_2-2 | 1.178 ms | 1.183 ms | +0.4% | | escrow-redeem_2-3 | 1.178 ms | 1.183 ms | +0.4% | | escrow-refund-1 | 814.0 μs | 792.9 μs | -2.6% | | future-increase-margin-1 | 786.8 μs | 786.5 μs | -0.0% | | future-increase-margin-2 | 1.368 ms | 1.185 ms | -13.4% | | future-increase-margin-3 | 1.366 ms | 1.186 ms | -13.2% | | future-increase-margin-4 | 2.032 ms | 2.029 ms | -0.1% | | future-increase-margin-5 | 2.535 ms | 2.554 ms | +0.7% | | future-pay-out-1 | 789.1 μs | 785.7 μs | -0.4% | | future-pay-out-2 | 1.365 ms | 1.186 ms | -13.1% | | future-pay-out-3 | 1.367 ms | 1.187 ms | -13.2% | | future-pay-out-4 | 2.541 ms | 2.557 ms | +0.6% | | future-settle-early-1 | 793.0 μs | 785.9 μs | -0.9% | | future-settle-early-2 | 1.367 ms | 1.186 ms | -13.2% | | future-settle-early-3 | 1.366 ms | 1.185 ms | -13.3% | | future-settle-early-4 | 2.205 ms | 2.213 ms | +0.4% | | game-sm-success_1-1 | 1.584 ms | 1.551 ms | -2.1% | | game-sm-success_1-2 | 588.0 μs | 591.5 μs | +0.6% | | game-sm-success_1-3 | 1.968 ms | 1.974 ms | +0.3% | | game-sm-success_1-4 | 633.9 μs | 641.5 μs | +1.2% | | game-sm-success_2-1 | 1.583 ms | 1.552 ms | -2.0% | | game-sm-success_2-2 | 588.8 μs | 591.8 μs | +0.5% | | game-sm-success_2-3 | 1.969 ms | 1.972 ms | +0.2% | | game-sm-success_2-4 | 634.8 μs | 641.8 μs | +1.1% | | game-sm-success_2-5 | 1.970 ms | 1.971 ms | +0.1% | | game-sm-success_2-6 | 635.3 μs | 641.2 μs | +0.9% | | multisig-sm-1 | 1.702 ms | 1.689 ms | -0.8% | | multisig-sm-2 | 1.702 ms | 1.668 ms | -2.0% | | multisig-sm-3 | 1.705 ms | 1.677 ms | -1.6% | | multisig-sm-4 | 1.716 ms | 1.678 ms | -2.2% | | multisig-sm-5 | 1.968 ms | 1.950 ms | -0.9% | | multisig-sm-6 | 1.718 ms | 1.680 ms | -2.2% | | multisig-sm-7 | 1.700 ms | 1.662 ms | -2.2% | | multisig-sm-8 | 1.708 ms | 1.693 ms | -0.9% | | multisig-sm-9 | 1.715 ms | 1.678 ms | -2.2% | | multisig-sm-10 | 1.968 ms | 1.950 ms | -0.9% | | ping-pong-1 | 1.420 ms | 1.387 ms | -2.3% | | ping-pong-2 | 1.421 ms | 1.386 ms | -2.5% | | ping-pong_2-1 | 1.212 ms | 1.179 ms | -2.7% | | prism-1 | 527.5 μs | 533.1 μs | +1.1% | | prism-2 | 1.597 ms | 1.583 ms | -0.9% | | prism-3 | 971.2 μs | 973.1 μs | +0.2% | | pubkey-1 | 502.8 μs | 496.6 μs | -1.2% | | stablecoin_1-1 | 3.135 ms | 3.093 ms | -1.3% | | stablecoin_1-2 | 577.9 μs | 596.5 μs | +3.2% | | stablecoin_1-3 | 3.394 ms | 3.341 ms | -1.6% | | stablecoin_1-4 | 597.2 μs | 603.1 μs | +1.0% | | stablecoin_1-5 | 3.859 ms | 3.896 ms | +1.0% | | stablecoin_1-6 | 672.1 μs | 675.2 μs | +0.5% | | stablecoin_2-1 | 3.156 ms | 3.098 ms | -1.8% | | stablecoin_2-2 | 578.4 μs | 585.7 μs | +1.3% | | stablecoin_2-3 | 3.368 ms | 3.341 ms | -0.8% | | stablecoin_2-4 | 597.5 μs | 602.3 μs | +0.8% | | token-account-1 | 711.3 μs | 692.6 μs | -2.6% | | token-account-2 | 869.7 μs | 887.5 μs | +2.0% | | uniswap-1 | 988.2 μs | 1.023 ms | +3.5% | | uniswap-2 | 755.2 μs | 747.5 μs | -1.0% | | uniswap-3 | 4.073 ms | 4.212 ms | +3.4% | | uniswap-4 | 807.3 μs | 814.4 μs | +0.9% | | uniswap-5 | 3.103 ms | 3.159 ms | +1.8% | | uniswap-6 | 786.7 μs | 791.2 μs | +0.6% | | vesting-1 | 1.106 ms | 1.106 ms | 0.0% |
5eb80f3 4a6c11a Change
TOTAL 112.8 ms 111.5 ms -1.2%
effectfully commented 1 day ago

WTF is going on in this PR

And it is consistent, here's the second output:

Screenshot from 2024-11-28 10-46-51

and the third one:

Screenshot from 2024-11-28 10-47-18

It appears that this PR somehow slows down the evaluator, while speeding up preparation, hence why we see results all over the place. How can it possibly slow down the evaluator? I have no idea, but this is likely me screwing something up, because I can't think of a way to make things slower by removing redundant code.

The experiment is over, so I'm closing this PR, but we need to make sure to investigate this weirdness in future.