garyschulte
commented
3 months ago Thanks for your benchmarking efforts. And thanks for highlighting the recent ECMUL regression - we were aware of the improvement in the typical use case, but were not aware of the regresson on those particular cases.
We have a few efforts underway to fix the regression, one is directly in besu:
Another is in besu-native:
As I am sure you are aware, benchmarks don't tell a complete or clear picture, but at least these tweaks should see besu perform better in the suites you have chosen.
It will be at least a release cycle before the first to improvements will be reflected in the benchmarks, but running locally these are the metrics I gathered for the ECMUL suite:
edit: I removed the reference to parallel tx processing, since the nethermind gas-benchmark suite appears to use a single transaction per block
CharlesFus
LukaszRozmej
Brief Description: This is our first Bounty, targeting performance optimization for the red items in the "Are We Fast Yet?"[^2]
[^2]: More background? Here: https://devlog.fusionist.io/posts/are-we-fast-yet/
Bounty Target: Based on today's result snapshot which I've converted into a single local HTML file, any
client + testcasewith a score greater than 10.0Bounty Details:
Theoretically, the maximum prize distribution for this event is: 2000 1 + 500 2 + 500 2 + 500 3 + 500 3 + 1500 3 + 1500 * 3 = 16000 USD
[^1]: The bounty will be paid in ACE, the native token of the Endurance network, equivalent to the USD amount, and transferred to the wallet address provided by the recipient.
Example Scenario:
Bob Improved Besu‘s "Burnt Pix" performance to a score of 5.0.
Bounty Calculation:
Given:
x1 = 10.0): 200 USDx2 = 1.0): 2000 USDLinear Interpolation Formula:
m = (2000 - 200) / (1 - 10) = -200b = 200 + 2000 = 2200y = -200 * 5 + 2200 = 1000 USDResult: Bob earns 1000 USD.
How to Prove Completion?
To claim a bounty, contributors must follow these steps:
Submit a PR: When submitting a Pull Request to the target project repository(Besu/Erigon/Geth/Nethermind/Reth), clearly state that the optimization is intended for a specific testcase in this bounty. This ensures the changes are targeted and not incidental.
Notify After PR Merge: Once the PR is merged and included in the latest release version of the client, notify us in this issue.
Verify Performance: I will review the latest results on arewefastyet to verify if the score is below 10.
Edge Cases:
Multiple Contributors: If multiple optimizations for a single testcase are included in one version, the bounty will be shared. The distribution of the bounty will be determined at our discretion.
Unclaimed Optimization: If no one claims an optimization but a testcase scores below 10.0 after a new version, the reward will typically be voided. However, I will review the commit history to determine if there indeed was an unintentional optimization by a developer, and if so, I will attempt to contact this guy to claim the reward.
Issue Closure:
This issue will close after six months or once all 7 testcases are optimized, whichever occurs first.
As this is our first time hosting a bounty, there may be unforeseen oversights. Therefore, the final interpretation of the rules and decisions related to this bounty remains at our discretion.