Closed GitGab19 closed 1 week ago
Branch | update-bencher-workflow |
Testbed | sv2 |
Branch | update-bencher-workflow |
Testbed | sv1 |
Branch | update-bencher-workflow |
Testbed | sv1 |
Benchmark | Estimated Cycles | Benchmark Result estimated cycles (Result Ξ%) | Upper Boundary estimated cycles (Limit %) | Instructions | Benchmark Result instructions (Result Ξ%) | Upper Boundary instructions (Limit %) | L1 Accesses | Benchmark Result accesses (Result Ξ%) | Upper Boundary accesses (Limit %) | L2 Accesses | Benchmark Result accesses (Result Ξ%) | Upper Boundary accesses (Limit %) | RAM Accesses | Benchmark Result accesses (Result Ξ%) | Upper Boundary accesses (Limit %) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
get_authorize | π view plot π· view threshold | 8,425.00 (-0.36%) | 8,830.86 (95.40%) | π view plot π· view threshold | 3,689.00 (-1.55%) | 3,892.73 (94.77%) | π view plot π· view threshold | 5,155.00 (-1.90%) | 5,462.75 (94.37%) | π view plot π· view threshold | 10.00 (+39.13%) | 12.94 (77.27%) | π view plot π· view threshold | 92.00 (+1.74%) | 96.22 (95.61%) |
get_submit | π view plot π· view threshold | 95,483.00 (+0.05%) | 96,078.05 (99.38%) | π view plot π· view threshold | 59,346.00 (-0.20%) | 59,768.99 (99.29%) | π view plot π· view threshold | 85,218.00 (-0.23%) | 85,855.52 (99.26%) | π view plot π· view threshold | 44.00 (-2.59%) | 64.62 (68.09%) | π view plot π· view threshold | 287.00 (+2.54%) | 289.60 (99.10%) |
get_subscribe | π view plot π· view threshold | 7,906.00 (-1.34%) | 8,388.12 (94.25%) | π view plot π· view threshold | 2,765.00 (-2.41%) | 2,972.69 (93.01%) | π view plot π· view threshold | 3,846.00 (-2.89%) | 4,155.95 (92.54%) | π view plot π· view threshold | 14.00 (+16.82%) | 20.69 (67.67%) | π view plot π· view threshold | 114.00 (-0.08%) | 120.16 (94.87%) |
serialize_authorize | π view plot π· view threshold | 12,248.00 (-0.21%) | 12,653.61 (96.79%) | π view plot π· view threshold | 5,266.00 (-0.99%) | 5,462.52 (96.40%) | π view plot π· view threshold | 7,323.00 (-1.27%) | 7,624.09 (96.05%) | π view plot π· view threshold | 12.00 (+32.30%) | 15.87 (75.61%) | π view plot π· view threshold | 139.00 (+1.12%) | 143.67 (96.75%) |
serialize_deserialize_authorize | π view plot π· view threshold | 24,412.00 (-1.09%) | 25,287.05 (96.54%) | π view plot π· view threshold | 9,838.00 (-0.53%) | 10,056.69 (97.83%) | π view plot π· view threshold | 13,887.00 (-0.54%) | 14,212.47 (97.71%) | π view plot π· view threshold | 33.00 (-5.04%) | 44.27 (74.55%) | π view plot π· view threshold | 296.00 (-1.75%) | 315.81 (93.73%) |
serialize_deserialize_handle_authorize | π view plot π· view threshold | 30,110.00 (-0.68%) | 30,796.22 (97.77%) | π view plot π· view threshold | 12,015.00 (-0.54%) | 12,227.44 (98.26%) | π view plot π· view threshold | 17,000.00 (-0.63%) | 17,322.01 (98.14%) | π view plot π· view threshold | 60.00 (+7.61%) | 68.38 (87.75%) | π view plot π· view threshold | 366.00 (-0.92%) | 381.93 (95.83%) |
serialize_deserialize_handle_submit | π view plot π· view threshold | 126,441.00 (-0.02%) | 127,148.01 (99.44%) | π view plot π· view threshold | 73,200.00 (-0.12%) | 73,652.02 (99.39%) | π view plot π· view threshold | 104,911.00 (-0.14%) | 105,611.17 (99.34%) | π view plot π· view threshold | 106.00 (-1.08%) | 133.59 (79.35%) | π view plot π· view threshold | 600.00 (+0.64%) | 605.62 (99.07%) |
serialize_deserialize_handle_subscribe | π view plot π· view threshold | 27,524.00 (-1.08%) | 28,650.81 (96.07%) | π view plot π· view threshold | 9,584.00 (-0.65%) | 9,791.58 (97.88%) | π view plot π· view threshold | 13,544.00 (-0.74%) | 13,852.97 (97.77%) | π view plot π· view threshold | 66.00 (+3.98%) | 76.03 (86.81%) | π view plot π· view threshold | 390.00 (-1.53%) | 417.22 (93.48%) |
serialize_deserialize_submit | π view plot π· view threshold | 115,163.00 (-0.06%) | 115,962.02 (99.31%) | π view plot π· view threshold | 68,060.00 (-0.06%) | 68,477.12 (99.39%) | π view plot π· view threshold | 97,653.00 (-0.09%) | 98,312.69 (99.33%) | π view plot π· view threshold | 65.00 (+3.42%) | 78.58 (82.72%) | π view plot π· view threshold | 491.00 (0.00%) | 499.69 (98.26%) |
serialize_deserialize_subscribe | π view plot π· view threshold | 22,914.00 (-1.42%) | 24,077.43 (95.17%) | π view plot π· view threshold | 8,143.00 (-0.72%) | 8,348.13 (97.54%) | π view plot π· view threshold | 11,464.00 (-0.78%) | 11,760.23 (97.48%) | π view plot π· view threshold | 36.00 (-3.54%) | 44.70 (80.54%) | π view plot π· view threshold | 322.00 (-2.02%) | 349.08 (92.24%) |
serialize_submit | π view plot π· view threshold | 99,872.00 (+0.02%) | 100,462.87 (99.41%) | π view plot π· view threshold | 61,408.00 (-0.16%) | 61,811.62 (99.35%) | π view plot π· view threshold | 88,077.00 (-0.20%) | 88,699.68 (99.30%) | π view plot π· view threshold | 49.00 (+4.27%) | 65.19 (75.16%) | π view plot π· view threshold | 330.00 (+1.68%) | 334.20 (98.74%) |
serialize_subscribe | π view plot π· view threshold | 11,351.00 (-0.39%) | 11,767.14 (96.46%) | π view plot π· view threshold | 4,118.00 (-1.50%) | 4,318.33 (95.36%) | π view plot π· view threshold | 5,706.00 (-1.91%) | 6,011.37 (94.92%) | π view plot π· view threshold | 16.00 (+24.73%) | 20.54 (77.90%) | π view plot π· view threshold | 159.00 (+0.93%) | 164.15 (96.86%) |
Branch | update-bencher-workflow |
Testbed | sv2 |
π¨ 5 ALERTS: Threshold Boundary Limits exceeded!
Benchmark | Measure Units | View | Benchmark Result (Result Ξ%) | Lower Boundary (Limit %) | Upper Boundary (Limit %) |
---|---|---|---|---|---|
client_sv2_mining_message_submit_standard_serialize_deserialize | Estimated Cycles estimated cycles | π plot π¨ alert π· threshold | 27,758.00 (+0.84%) | 27,742.47 (100.06%) | |
client_sv2_mining_message_submit_standard_serialize_deserialize | Instructions instructions | π plot π¨ alert π· threshold | 10,645.00 (+0.62%) | 10,643.34 (100.02%) | |
client_sv2_mining_message_submit_standard_serialize_deserialize | L1 Accesses accesses | π plot π¨ alert π· threshold | 15,503.00 (+0.71%) | 15,494.64 (100.05%) | |
client_sv2_mining_message_submit_standard_serialize_deserialize | L2 Accesses accesses | π plot π¨ alert π· threshold | 92.00 (+12.97%) | 90.41 (101.76%) | |
client_sv2_open_channel_serialize_deserialize | L2 Accesses accesses | π plot π¨ alert π· threshold | 87.00 (+16.90%) | 85.60 (101.64%) |
Benchmark | Estimated Cycles | Benchmark Result estimated cycles (Result Ξ%) | Upper Boundary estimated cycles (Limit %) | Instructions | Benchmark Result instructions (Result Ξ%) | Upper Boundary instructions (Limit %) | L1 Accesses | Benchmark Result accesses (Result Ξ%) | Upper Boundary accesses (Limit %) | L2 Accesses | Benchmark Result accesses (Result Ξ%) | Upper Boundary accesses (Limit %) | RAM Accesses | Benchmark Result accesses (Result Ξ%) | Upper Boundary accesses (Limit %) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
client_sv2_handle_message_common | π view plot π· view threshold | 2,099.00 (+0.32%) | 2,203.56 (95.25%) | π view plot π· view threshold | 473.00 (-0.04%) | 488.74 (96.78%) | π view plot π· view threshold | 734.00 (-0.23%) | 757.85 (96.85%) | π view plot π· view threshold | 7.00 (+27.64%) | 14.24 (49.15%) | π view plot π· view threshold | 38.00 (+0.06%) | 41.09 (92.47%) |
client_sv2_handle_message_mining | π view plot π· view threshold | 8,166.00 (-0.47%) | 8,289.33 (98.51%) | π view plot π· view threshold | 2,137.00 (-0.03%) | 2,143.08 (99.72%) | π view plot π· view threshold | 3,161.00 (+0.01%) | 3,170.58 (99.70%) | π view plot π· view threshold | 35.00 (-1.58%) | 42.23 (82.88%) | π view plot π· view threshold | 138.00 (-0.74%) | 140.96 (97.90%) |
client_sv2_mining_message_submit_standard | π view plot π· view threshold | 6,289.00 (+0.17%) | 6,386.95 (98.47%) | π view plot π· view threshold | 1,750.00 (-0.04%) | 1,767.13 (99.03%) | π view plot π· view threshold | 2,549.00 (-0.18%) | 2,576.15 (98.95%) | π view plot π· view threshold | 20.00 (+17.49%) | 24.77 (80.75%) | π view plot π· view threshold | 104.00 (+0.02%) | 106.96 (97.23%) |
client_sv2_mining_message_submit_standard_serialize | π view plot π· view threshold | 14,732.00 (+0.01%) | 14,965.15 (98.44%) | π view plot π· view threshold | 4,694.00 (-0.02%) | 4,711.13 (99.64%) | π view plot π· view threshold | 6,752.00 (-0.10%) | 6,782.87 (99.54%) | π view plot π· view threshold | 49.00 (+12.48%) | 53.66 (91.32%) | π view plot π· view threshold | 221.00 (-0.25%) | 227.71 (97.05%) |
client_sv2_mining_message_submit_standard_serialize_deserialize | π view plot π¨ view alert π· view threshold | 27,758.00 (+0.84%) | 27,742.47 (100.06%) | π view plot π¨ view alert π· view threshold | 10,645.00 (+0.62%) | 10,643.34 (100.02%) | π view plot π¨ view alert π· view threshold | 15,503.00 (+0.71%) | 15,494.64 (100.05%) | π view plot π¨ view alert π· view threshold | 92.00 (+12.97%) | 90.41 (101.76%) | π view plot π· view threshold | 337.00 (+0.59%) | 341.28 (98.75%) |
client_sv2_open_channel | π view plot π· view threshold | 4,417.00 (+0.45%) | 4,611.96 (95.77%) | π view plot π· view threshold | 1,461.00 (-0.01%) | 1,476.74 (98.93%) | π view plot π· view threshold | 2,157.00 (-0.12%) | 2,185.41 (98.70%) | π view plot π· view threshold | 11.00 (+22.22%) | 17.77 (61.91%) | π view plot π· view threshold | 63.00 (+0.56%) | 68.39 (92.12%) |
client_sv2_open_channel_serialize | π view plot π· view threshold | 14,024.00 (-0.26%) | 14,448.77 (97.06%) | π view plot π· view threshold | 5,064.00 (-0.00%) | 5,079.74 (99.69%) | π view plot π· view threshold | 7,324.00 (-0.02%) | 7,353.16 (99.60%) | π view plot π· view threshold | 38.00 (+8.06%) | 43.43 (87.50%) | π view plot π· view threshold | 186.00 (-0.75%) | 198.59 (93.66%) |
client_sv2_open_channel_serialize_deserialize | π view plot π· view threshold | 22,866.00 (+0.89%) | 22,907.78 (99.82%) | π view plot π· view threshold | 8,040.00 (+0.25%) | 8,074.89 (99.57%) | π view plot π· view threshold | 11,686.00 (+0.17%) | 11,755.22 (99.41%) | π view plot π¨ view alert π· view threshold | 87.00 (+16.90%) | 85.60 (101.64%) | π view plot π· view threshold | 307.00 (+1.13%) | 311.80 (98.46%) |
client_sv2_setup_connection | π view plot π· view threshold | 4,689.00 (+0.11%) | 4,771.48 (98.27%) | π view plot π· view threshold | 1,502.00 (-0.01%) | 1,517.74 (98.96%) | π view plot π· view threshold | 2,274.00 (-0.19%) | 2,299.53 (98.89%) | π view plot π· view threshold | 14.00 (+49.58%) | 15.92 (87.93%) | π view plot π· view threshold | 67.00 (-0.58%) | 69.80 (95.99%) |
client_sv2_setup_connection_serialize | π view plot π· view threshold | 16,164.00 (-0.06%) | 16,466.01 (98.17%) | π view plot π· view threshold | 5,963.00 (-0.00%) | 5,978.74 (99.74%) | π view plot π· view threshold | 8,659.00 (-0.05%) | 8,692.74 (99.61%) | π view plot π· view threshold | 45.00 (+12.43%) | 53.63 (83.90%) | π view plot π· view threshold | 208.00 (-0.40%) | 216.49 (96.08%) |
client_sv2_setup_connection_serialize_deserialize | π view plot π· view threshold | 35,704.00 (+0.43%) | 35,747.34 (99.88%) | π view plot π· view threshold | 14,888.00 (+0.27%) | 14,906.81 (99.87%) | π view plot π· view threshold | 21,879.00 (+0.31%) | 21,918.42 (99.82%) | π view plot π· view threshold | 98.00 (+5.47%) | 114.39 (85.67%) | π view plot π· view threshold | 381.00 (+0.46%) | 384.65 (99.05%) |
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 19.30%. Comparing base (
b900d0a
) to head (c54af5e
). Report is 2 commits behind head on main.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
This PR updates the workflows used to run benchmarks on our CI. According to @epompeii, we need to add
permissions: checks: write
to solve errors we are getting when merging PRs tomain
.This PR also adds a new workflow to automatically archive benches from forks after PRs are merged/closed.