Closed msmouse closed 2 weeks ago
🚨 3 jobs on the last run were significantly faster/slower than expected
Job | Duration | vs 7d avg | Delta |
---|---|---|---|
forge-e2e-test / forge | 22m | 15m | |
windows-build | 30m | 39m | |
forge-framework-upgrade-test / forge | 18m | 1h 8m |
main
This stack of pull requests is managed by Graphite. Learn more about stacking.
Join @msmouse and the rest of your teammates on Graphite
compat
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 129e3230b237b048bf66bbdb0f43d39c6a9c281c
realistic_env_max_load
on 129e3230b237b048bf66bbdb0f43d39c6a9c281c
compat
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 129e3230b237b048bf66bbdb0f43d39c6a9c281c
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 129e3230b237b048bf66bbdb0f43d39c6a9c281c (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6439.871125402387 txn/s, latency: 5065.986596203624 ms, (p50: 5000 ms, p90: 8400 ms, p99: 10100 ms), latency samples: 231800
2. Upgrading first Validator to new version: 129e3230b237b048bf66bbdb0f43d39c6a9c281c
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1654.4089573744764 txn/s, latency: 16325.045850340137 ms, (p50: 19100 ms, p90: 23800 ms, p99: 24400 ms), latency samples: 88200
3. Upgrading rest of first batch to new version: 129e3230b237b048bf66bbdb0f43d39c6a9c281c
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1617.6307493021209 txn/s, latency: 17701.38355172414 ms, (p50: 19500 ms, p90: 23600 ms, p99: 24100 ms), latency samples: 87000
4. upgrading second batch to new version: 129e3230b237b048bf66bbdb0f43d39c6a9c281c
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3542.7446036810065 txn/s, latency: 8838.532672855757 ms, (p50: 9600 ms, p90: 12700 ms, p99: 13100 ms), latency samples: 144340
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 129e3230b237b048bf66bbdb0f43d39c6a9c281c passed
Test Ok
realistic_env_max_load
success on 129e3230b237b048bf66bbdb0f43d39c6a9c281c
two traffics test: inner traffic : committed: 7987.949049952605 txn/s, latency: 4902.496202557279 ms, (p50: 4700 ms, p90: 5700 ms, p99: 10800 ms), latency samples: 3456800
two traffics test : committed: 100.05028316425756 txn/s, latency: 1937.8427777777779 ms, (p50: 1900 ms, p90: 2100 ms, p99: 5700 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.206, avg: 0.203", "QsPosToProposal: max: 0.288, avg: 0.234", "ConsensusProposalToOrdered: max: 0.483, avg: 0.429", "ConsensusOrderedToCommit: max: 0.389, avg: 0.371", "ConsensusProposalToCommit: max: 0.817, avg: 0.800"]
Max round gap was 1 [limit 4] at version 1700657. Max no progress secs was 5.056364 [limit 15] at version 1700657.
Test Ok
realistic_env_max_load
on 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
compat
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
framework_upgrade
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
realistic_env_max_load
success on 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
two traffics test: inner traffic : committed: 7713.9725264360795 txn/s, latency: 5070.735118216834 ms, (p50: 4800 ms, p90: 6000 ms, p99: 11100 ms), latency samples: 3338780
two traffics test : committed: 100.04563816883315 txn/s, latency: 1916.7811764705882 ms, (p50: 1900 ms, p90: 2200 ms, p99: 3100 ms), latency samples: 1700
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.206, avg: 0.201", "QsPosToProposal: max: 0.286, avg: 0.234", "ConsensusProposalToOrdered: max: 0.449, avg: 0.424", "ConsensusOrderedToCommit: max: 0.418, avg: 0.378", "ConsensusProposalToCommit: max: 0.854, avg: 0.802"]
Max round gap was 1 [limit 4] at version 1621942. Max no progress secs was 4.551866 [limit 15] at version 1621942.
Test Ok
compat
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 46c5eaffa52b94ea0b75e4c0453c906fc98471ad (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 2604.7045445332374 txn/s, latency: 10014.223490878938 ms, (p50: 9000 ms, p90: 13500 ms, p99: 18700 ms), latency samples: 120600
2. Upgrading first Validator to new version: 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1230.0353695305096 txn/s, latency: 20633.656787452677 ms, (p50: 25700 ms, p90: 29300 ms, p99: 31400 ms), latency samples: 73960
3. Upgrading rest of first batch to new version: 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1236.5747473406616 txn/s, latency: 21564.999073327963 ms, (p50: 25500 ms, p90: 30300 ms, p99: 31000 ms), latency samples: 74460
4. upgrading second batch to new version: 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 2880.427208916277 txn/s, latency: 10258.166718775712 ms, (p50: 9900 ms, p90: 13200 ms, p99: 16100 ms), latency samples: 121540
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 46c5eaffa52b94ea0b75e4c0453c906fc98471ad passed
Test Ok
framework_upgrade
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 46c5eaffa52b94ea0b75e4c0453c906fc98471ad (PR)
Upgrade the nodes to version: 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1420.8833650003266 txn/s, submitted: 1423.1073563542402 txn/s, failed submission: 2.2239913539135547 txn/s, expired: 2.2239913539135547 txn/s, latency: 2379.3135739130435 ms, (p50: 1800 ms, p90: 4200 ms, p99: 5500 ms), latency samples: 115000
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1167.7634052169005 txn/s, submitted: 1170.6599753882074 txn/s, failed submission: 2.8965701713069465 txn/s, expired: 2.8965701713069465 txn/s, latency: 2657.277800038161 ms, (p50: 2100 ms, p90: 4700 ms, p99: 6600 ms), latency samples: 104820
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 46c5eaffa52b94ea0b75e4c0453c906fc98471ad passed
Upgrade the remaining nodes to version: 46c5eaffa52b94ea0b75e4c0453c906fc98471ad
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1113.9785629459582 txn/s, submitted: 1117.3038720890804 txn/s, failed submission: 3.3253091431222632 txn/s, expired: 3.3253091431222632 txn/s, latency: 2698.228855721393 ms, (p50: 2400 ms, p90: 4400 ms, p99: 5700 ms), latency samples: 100500
Test Ok
framework_upgrade
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 265eddefd788e5b2163aed48e048bee0afcf8347
compat
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 265eddefd788e5b2163aed48e048bee0afcf8347
realistic_env_max_load
on 265eddefd788e5b2163aed48e048bee0afcf8347
compat
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 265eddefd788e5b2163aed48e048bee0afcf8347
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 265eddefd788e5b2163aed48e048bee0afcf8347 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6494.182257092332 txn/s, latency: 5138.318560539688 ms, (p50: 5100 ms, p90: 8500 ms, p99: 9800 ms), latency samples: 228280
2. Upgrading first Validator to new version: 265eddefd788e5b2163aed48e048bee0afcf8347
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1831.6744484917554 txn/s, latency: 15697.529438642297 ms, (p50: 19600 ms, p90: 22200 ms, p99: 22600 ms), latency samples: 91920
3. Upgrading rest of first batch to new version: 265eddefd788e5b2163aed48e048bee0afcf8347
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1811.0270256098354 txn/s, latency: 15856.045523370638 ms, (p50: 18700 ms, p90: 22300 ms, p99: 22700 ms), latency samples: 91140
4. upgrading second batch to new version: 265eddefd788e5b2163aed48e048bee0afcf8347
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3246.9982967668334 txn/s, latency: 9230.78892228739 ms, (p50: 9900 ms, p90: 11600 ms, p99: 12100 ms), latency samples: 136400
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 265eddefd788e5b2163aed48e048bee0afcf8347 passed
Test Ok
framework_upgrade
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 265eddefd788e5b2163aed48e048bee0afcf8347
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 265eddefd788e5b2163aed48e048bee0afcf8347 (PR)
Upgrade the nodes to version: 265eddefd788e5b2163aed48e048bee0afcf8347
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1288.2988971947655 txn/s, submitted: 1291.2641744014813 txn/s, failed submission: 2.965277206715997 txn/s, expired: 2.965277206715997 txn/s, latency: 2380.973539305949 ms, (p50: 2100 ms, p90: 4100 ms, p99: 5700 ms), latency samples: 112960
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1152.112291303082 txn/s, submitted: 1154.3097208115753 txn/s, failed submission: 2.1974295084933853 txn/s, expired: 2.1974295084933853 txn/s, latency: 2673.242504291436 ms, (p50: 2100 ms, p90: 4400 ms, p99: 7200 ms), latency samples: 104860
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 265eddefd788e5b2163aed48e048bee0afcf8347 passed
Upgrade the remaining nodes to version: 265eddefd788e5b2163aed48e048bee0afcf8347
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1203.9985160883327 txn/s, submitted: 1205.1478082953663 txn/s, failed submission: 1.1492922070335363 txn/s, expired: 1.1492922070335363 txn/s, latency: 2598.5283791523484 ms, (p50: 2100 ms, p90: 4800 ms, p99: 6900 ms), latency samples: 104760
Test Ok
realistic_env_max_load
success on 265eddefd788e5b2163aed48e048bee0afcf8347
two traffics test: inner traffic : committed: 7889.9759036790065 txn/s, latency: 4967.749417107941 ms, (p50: 4800 ms, p90: 5700 ms, p99: 10500 ms), latency samples: 3410580
two traffics test : committed: 99.98344951601374 txn/s, latency: 1907.3103448275863 ms, (p50: 1800 ms, p90: 2100 ms, p99: 6900 ms), latency samples: 1740
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.206, avg: 0.202", "QsPosToProposal: max: 0.230, avg: 0.216", "ConsensusProposalToOrdered: max: 0.440, avg: 0.416", "ConsensusOrderedToCommit: max: 0.387, avg: 0.354", "ConsensusProposalToCommit: max: 0.823, avg: 0.770"]
Max round gap was 1 [limit 4] at version 266273. Max no progress secs was 4.925562 [limit 15] at version 2761075.
Test Ok
In the presence of deletion, a leaf node on the SMT can be lifted, so the hash value won't be matching that in the DB.
Examining more, it's obvious that in such cases proof fetching is not needed in the first place.
Type of Change
Which Components or Systems Does This Change Impact?
How Has This Been Tested?
updated unit tests. replay-verify.
it's unfortunate unit tests don't have much coverage for state deletion -- a tech debt I'm gonna fix as a follow up.
Key Areas to Review
Checklist