Closed sifislag closed 1 year ago
41 tests ±0 41 :heavy_check_mark: +1 17m 45s :stopwatch: -9s 1 suites ±0 0 :zzz: ±0 1 files ±0 0 :x: - 1
Results for commit 9c9f09ca. ± Comparison against base commit d7b00c1b.
:recycle: This comment has been updated with latest results.
41 tests ±0 41 :heavy_check_mark: +1 26m 15s :stopwatch: + 2m 38s 1 suites ±0 0 :zzz: ±0 1 files ±0 0 :x: - 1
Results for commit 9c9f09ca. ± Comparison against base commit d7b00c1b.
:recycle: This comment has been updated with latest results.
For a dataset of 1022 viaIR contracts and a timeout limit of 800s per contract:
python3.10 tooling/compare-runs.py newdefaults-noclients-old.json newdefaults-noclients-new.json --decomp
1022 total contracts
957 contracts decompiled/analyzed by newdefaults-noclients-old (0 exclusively)
370 with the default config, 520 with the precise, 67 with the scalable fallback
1021 contracts decompiled/analyzed by newdefaults-noclients-new (64 exclusively)
974 with the default config, 47 with the scalable fallback
1021 contracts decompiled/analyzed by some config
957 contracts decompiled/analyzed by all configs (common)
ANALYTIC: decomp_time
newdefaults-noclients-old (common): 46527.27614617348 (+211.2%)
newdefaults-noclients-new (common): 14951.580310583115
ANALYTIC: Analytics_JumpToMany
newdefaults-noclients-old (common): 3567 (+62.43%)
newdefaults-noclients-new (common): 2196
ANALYTIC: Analytics_ReachableBlocks
newdefaults-noclients-old (common): 449701
newdefaults-noclients-new (common): 448458 (-0.2764%)
ANALYTIC: Analytics_UnreachableBlock
newdefaults-noclients-old (common): 17370
newdefaults-noclients-new (common): 18613 (+7.156%)
ANALYTIC: Analytics_ReachableBlocksInTAC
newdefaults-noclients-old (common): 448555
newdefaults-noclients-new (common): 447908 (-0.1442%)
ANALYTIC: Analytics_BlockHasNoTACBlock
newdefaults-noclients-old (common): 1146 (+108.4%)
newdefaults-noclients-new (common): 550
ANALYTIC: Analytics_DeadBlocks
newdefaults-noclients-old (common): 69903 (+2020%)
newdefaults-noclients-new (common): 3298
ANALYTIC: Analytics_PolymorphicTargetSameCtx
newdefaults-noclients-old (common): 3002 (+1343%)
newdefaults-noclients-new (common): 208
ANALYTIC: Analytics_LocalBlockEdge
newdefaults-noclients-old (common): 656823
newdefaults-noclients-new (common): 637568 (-2.932%)
ANALYTIC: Analytics_Contexts
newdefaults-noclients-old (common): 3051674 (+160.2%)
newdefaults-noclients-new (common): 1172618
Same dataset, a timeout of 400s no scalable fallback config (also no precise fallback for the old defaults):
1022 total contracts
890 contracts decompiled/analyzed by newdefaults-onlydecomp-old (11 exclusively)
974 contracts decompiled/analyzed by newdefaults-onlydecomp-new (95 exclusively)
985 contracts decompiled/analyzed by some config
879 contracts decompiled/analyzed by all configs (common)
890 974
ANALYTIC: decomp_time
newdefaults-onlydecomp-old (common): 4787.565344572067 (+16.52%)
newdefaults-onlydecomp-new (common): 4108.722040176392
ANALYTIC: Analytics_JumpToMany
newdefaults-onlydecomp-old (common): 3264 (+96.51%)
newdefaults-onlydecomp-new (common): 1661
ANALYTIC: Analytics_ReachableBlocks
newdefaults-onlydecomp-old (common): 395088
newdefaults-onlydecomp-new (common): 395088
ANALYTIC: Analytics_UnreachableBlock
newdefaults-onlydecomp-old (common): 16464
newdefaults-onlydecomp-new (common): 16464
ANALYTIC: Analytics_ReachableBlocksInTAC
newdefaults-onlydecomp-old (common): 394247 (-0.08566%)
newdefaults-onlydecomp-new (common): 394585
ANALYTIC: Analytics_BlockHasNoTACBlock
newdefaults-onlydecomp-old (common): 841 (+67.2%)
newdefaults-onlydecomp-new (common): 503
ANALYTIC: Analytics_DeadBlocks
newdefaults-onlydecomp-old (common): 8752 (+235.5%)
newdefaults-onlydecomp-new (common): 2609
ANALYTIC: Analytics_PolymorphicTargetSameCtx
newdefaults-onlydecomp-old (common): 1174 (+846.8%)
newdefaults-onlydecomp-new (common): 124
ANALYTIC: Analytics_LocalBlockEdge
newdefaults-onlydecomp-old (common): 563354
newdefaults-onlydecomp-new (common): 561414 (-0.3444%)
ANALYTIC: Analytics_Contexts
newdefaults-onlydecomp-old (common): 1989144 (+130.8%)
newdefaults-onlydecomp-new (common): 861755
This PR includes major changes to the default gigahorse decompilation pipeline:
--early_cloning
by default after the latest optimizationsFiniteShrinkingContext
Will update with experimental results.