chore: ABCI methods benchmarks

celestiaorg / celestia-app

Celestia consensus node

https://celestiaorg.github.io/celestia-app/

Apache License 2.0

345 stars 292 forks source link

chore: ABCI methods benchmarks #3904

Closed rach-id closed 1 month ago

rach-id commented 1 month ago

Works on this: https://github.com/celestiaorg/celestia-app/issues/3898

To run the benchmarks, run the following in the root directory:

go test -bench=<benchmark_name> app/benchmark_test.go

with <benchmark_name> is:

BenchmarkCheckTx_MsgSend_1: for a benchmark of a MsgSend transaction in CheckTx
BenchmarkCheckTx_MsgSend_8MB: for a benchmark of 8mb block worth of MsgSend transactions in CheckTx
BenchmarkDeliverTx_MsgSend_1: for a benchmark of a MsgSend transaction in DeliverTx
BenchmarkDeliverTx_MsgSend_8MB: for a benchmark of 8mb block worth of MsgSend transactions in DeliverTx
BenchmarkPrepareProposal_MsgSend_1: for a benchmark of a block containing a singleMsgSend transaction in PrepareProposal
BenchmarkPrepareProposal_MsgSend_8MB: for a benchmark of an 8mb block containing MsgSend transactions in PrepareProposal
BenchmarkProcessProposal_MsgSend_1: for a benchmark of a block containing a singleMsgSend transaction in ProcessProposal
BenchmarkProcessProposal_MsgSend_8MB: for a benchmark of an 8mb block containingMsgSend transactions in ProcessProposal
...

Note: keeping this as a draft because it doesn't necessarily need to be merged

Benchmark run on: Macbook pro M3 max 48GB RAM

coderabbitai[bot] commented 1 month ago

[!NOTE]

Reviews paused

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.

@coderabbitai review to trigger a single review.

📝 Walkthrough

## Walkthrough This pull request introduces a suite of benchmark tests for transaction processing functionalities in the Celestia application, focusing on `MsgSend`, IBC client updates, and "Pay For Blob" transactions. It updates constants related to transaction limits and modifies the test application initialization for improved timestamp accuracy. New utility functions for transaction generation and performance metrics calculation are also added. ## Changes | Files | Change Summary | |-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `app/benchmarks/benchmark_ibc_update_client_test.go`, `app/benchmarks/benchmark_msg_send_test.go`, `app/benchmarks/benchmark_pfb_test.go` | Added benchmark tests for IBC client updates, MsgSend transactions, and PFB transactions, including various performance measurement functions and utility functions for transaction generation and metrics calculation. | | `pkg/appconsts/initial_consts.go`, `pkg/appconsts/global_consts.go`, `pkg/appconsts/prepare_proposal_consts.go`, `pkg/appconsts/prepare_proposal_consts_bench.go` | Updated `DefaultGovMaxSquareSize`, removed old transaction cap constants, and introduced new transaction cap constants with specified limits. Added a benchmarking file for transaction caps. | | `test/util/test_app.go` | Enhanced initialization of the test application, restructuring `GenesisTime`, adding a logger, and improving the dynamic assignment of block header timestamps. | ## Possibly related PRs - **#3942**: This PR introduces limits on the number of transactions in the prepare proposal phase, which is directly related to the changes in the main PR that also involve transaction processing and preparation. - **#3909**: This PR sets a maximum transaction size, which aligns with the main PR's focus on transaction processing limits and performance evaluation. - **#3967**: This PR refactors transaction size handling, renaming `MaxTxBytes` to `MaxTxSize`, which is relevant to the main PR's changes regarding transaction limits and processing logic. ## Suggested labels `testing`, `enhancement`, `WS: Big Blonks 🔭` ## Suggested reviewers - evan-forbes - staheri14 - ninabarbakadze - cmwaters

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

🪧 Tips

### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): > ‼️ **IMPORTANT** > Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged. - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` - `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (Invoked using PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. ### Other keywords and placeholders - Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. - Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description. - Add `@coderabbitai` anywhere in the PR title to generate the title automatically. ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

rach-id commented 1 month ago

First results:

BenchmarkCheckTx_MsgSend:
```
5904            169616 ns/op
```
BenchmarkDeliverTx_MsgSend:
```
22509             53807 ns/op
```
BenchmarkPrepareProposal_MsgSend_1:
```
4668            222538 ns/op
```

BenchmarkPrepareProposal_MsgSend_8MB:


I[2024-09-25|10:31:46.555] block prepared                               numberoftransactions=31645 blocksize(mb)~=7.408

BenchmarkPrepareProposal_MsgSend_8MB-16 1 6109720625 ns/op

which contains 31645 msg send transactions in a block totalling 7.4mb

- `BenchmarkProcessProposal_MsgSend_1`:

4855 215620 ns/op


- `BenchmarkProcessProposal_MsgSend_8MB`:

I[2024-09-25|10:34:39.983] block prepared numberoftransactions=31645 blocksize(mb)~=7.408

BenchmarkProcessProposal_MsgSend_8MB-16 1 4891330792 ns/op


which contains 31645 msg send transactions in a block totalling 7.4mb

cc @cmwaters

rach-id commented 1 month ago

Concerning the second part of the issue which requires testing the update client transaction. Does it make sense to instead test a multisig send transaction and have the same number of signatures (150 for example). This would be the same cost if we just want to check how costly it is to verify the signatures.

cc @cmwaters @evan-forbes

cmwaters commented 1 month ago

Is this just because it's far simpler to do that than create the IBC headers? I think a multi sig can only have at most 7 transactions

cmwaters commented 1 month ago

Am I reading this correctly that it takes 6 seconds for prepare proposal and almost 5 seconds for process proposal? (at 8MB of SendMsgs)

That's quite a bit.

Can you also measure how long it takes to have 8MB worth of send msgs via DeliverTx - or can I just go 31645 * 53807 ns/op?

Do you know why CheckTx takes more than 3 times longer than DeliverTx?

And then finally it would be good to compare this with 8MB worth of blob transactions that are each 4KB in size (number picked at random - I actually don't know what the average PFB size is).

rach-id commented 1 month ago

Am I reading this correctly that it takes 6 seconds for prepare proposal and almost 5 seconds for process proposal? (at 8MB of SendMsgs)

yes.

Can you also measure how long it takes to have 8MB worth of send msgs via DeliverTx - or can I just go 31645 * 53807 ns/op?

Added bench of 8mb worth of msg send for CheckTx in BenchmarkCheckTx_MsgSend_8MB and results are:
```
BenchmarkCheckTx_MsgSend_8MB-16            1    6674640667 ns/op
```
Also, for DeliverTx, with same number of transactions in BenchmarkDeliverTx_MsgSend_8MB:
```
BenchmarkDeliverTx_MsgSend_8MB-16              1    2114091208 ns/op
```

Will work on the points and get you the results.

cmwaters commented 1 month ago

Ok so that seems like a significant amount of time given we want blocks to be sub 6 seconds.

If we're going to limit the gas in a block can you also work out the total gas of those two respective blocks I.e. (all MsgSend and all 4KB PFBs) 🙏

rach-id commented 1 month ago

For the PFBs, I added the following benchmarks: BenchmarkPrepareProposal_PFB_Multi which benchmarks prepare proposal with blocks with different blob sizes. The results are:

15000_transactions_of_300_bytes-16                 1    2501182208 ns/op             6.239 block_size(mb)        10318 number_of_transactions   2501182208 prepare_proposal_time(ns)    988490895000 total_gas_used        703.0 transactions_size(byte)

10000_transactions_of_500_bytes-16                 1    1689812542 ns/op             5.035 block_size(mb)         6331 number_of_transactions   1689812542 prepare_proposal_time(ns)    439343930000 total_gas_used        903.0 transactions_size(byte)

6000_transactions_of_1000_bytes-16                 1    1036703375 ns/op             5.809 block_size(mb)         4566 number_of_transactions   1036703375 prepare_proposal_time(ns)    158174358000 total_gas_used       1403 transactions_size(byte)

3000_transactions_of_5000_bytes-16          1000000000           0.5361 ns/op            7.188 block_size(mb)         1413 number_of_transactions    536124042 prepare_proposal_time(ns)    39550179000 total_gas_used        5403 transactions_size(byte)

1000_transactions_of_10000_bytes-16         1000000000           0.2098 ns/op            7.470 block_size(mb)          758.0 number_of_transactions  209832875 prepare_proposal_time(ns)    4397393000 total_gas_used        10403 transactions_size(byte)

500_transactions_of_50000_bytes-16          1000000000           0.1262 ns/op            7.441 block_size(mb)          155.0 number_of_transactions  126233584 prepare_proposal_time(ns)    1100446500 total_gas_used        50406 transactions_size(byte)

100_transactions_of_100000_bytes-16         1000000000           0.05864 ns/op           7.368 block_size(mb)           77.00 number_of_transactions      58644708 prepare_proposal_time(ns)      44369300 total_gas_used       100406 transactions_size(byte)

100_transactions_of_200000_bytes-16         1000000000           0.06557 ns/op           7.260 block_size(mb)           38.00 number_of_transactions      65569750 prepare_proposal_time(ns)      44369300 total_gas_used       200406 transactions_size(byte)

50_transactions_of_300000_bytes-16          1000000000           0.05194 ns/op           7.161 block_size(mb)           25.00 number_of_transactions      51936375 prepare_proposal_time(ns)      11202150 total_gas_used       300406 transactions_size(byte)

50_transactions_of_400000_bytes-16          1000000000           0.04908 ns/op           7.254 block_size(mb)           19.00 number_of_transactions      49077375 prepare_proposal_time(ns)      11202150 total_gas_used       400406 transactions_size(byte)

30_transactions_of_500000_bytes-16          1000000000           0.05085 ns/op           7.157 block_size(mb)           15.00 number_of_transactions      50846708 prepare_proposal_time(ns)       4085490 total_gas_used       500406 transactions_size(byte)

10_transactions_of_1000000_bytes-16         1000000000           0.03060 ns/op           6.678 block_size(mb)            7.000 number_of_transactions     30597791 prepare_proposal_time(ns)        483230 total_gas_used      1000406 transactions_size(byte)

5_transactions_of_2000000_bytes-16          1000000000           0.02862 ns/op           5.723 block_size(mb)            3.000 number_of_transactions     28619250 prepare_proposal_time(ns)        131790 total_gas_used      2000406 transactions_size(byte)

3_transactions_of_3000000_bytes-16          1000000000           0.04402 ns/op           5.723 block_size(mb)            2.000 number_of_transactions     44022417 prepare_proposal_time(ns)         52716 total_gas_used      3000409 transactions_size(byte)

3_transactions_of_4000000_bytes-16          1000000000           0.03565 ns/op           3.815 block_size(mb)            1.000 number_of_transactions     35649792 prepare_proposal_time(ns)         52716 total_gas_used      4000409 transactions_size(byte)

2_transactions_of_5000000_bytes-16          1000000000           0.03975 ns/op           4.769 block_size(mb)            1.000 number_of_transactions     39753500 prepare_proposal_time(ns)         26358 total_gas_used      5000409 transactions_size(byte)

2_transactions_of_6000000_bytes-16          1000000000           0.03288 ns/op           5.722 block_size(mb)            1.000 number_of_transactions     32875500 prepare_proposal_time(ns)         26358 total_gas_used      6000409 transactions_size(byte)

cc @evan-forbes @cmwaters

rach-id commented 1 month ago

Same for process proposal, I added BenchmarkProcessProposal_PFB_Multi which benchmarks process proposal using different blob sizes. The results:

15000_transactions_of_300_bytes-16                 1    1689482458 ns/op             6.239 block_size(mb)        10318 number_of_transactions   1689482458 process_proposal_time(ns)    988490895000 total_gas_used        703.0 transactions_size(byte)

10000_transactions_of_500_bytes-16                 1    1079362750 ns/op             5.035 block_size(mb)         6331 number_of_transactions   1079362750 process_proposal_time(ns)    439343930000 total_gas_used        903.0 transactions_size(byte)

6000_transactions_of_1000_bytes-16          1000000000           0.8345 ns/op            5.809 block_size(mb)         4566 number_of_transactions    834506584 process_proposal_time(ns)    158174358000 total_gas_used       1403 transactions_size(byte)

3000_transactions_of_5000_bytes-16          1000000000           0.2946 ns/op            7.188 block_size(mb)         1413 number_of_transactions    294579000 process_proposal_time(ns)    39550179000 total_gas_used        5403 transactions_size(byte)

1000_transactions_of_10000_bytes-16         1000000000           0.1982 ns/op            7.470 block_size(mb)          758.0 number_of_transactions  198208000 process_proposal_time(ns)    4397393000 total_gas_used        10403 transactions_size(byte)

500_transactions_of_50000_bytes-16          1000000000           0.08576 ns/op           7.441 block_size(mb)          155.0 number_of_transactions   85762584 process_proposal_time(ns)    1100446500 total_gas_used        50406 transactions_size(byte)

100_transactions_of_100000_bytes-16         1000000000           0.06285 ns/op           7.368 block_size(mb)           77.00 number_of_transactions      62845333 process_proposal_time(ns)      44369300 total_gas_used       100406 transactions_size(byte)

100_transactions_of_200000_bytes-16         1000000000           0.06041 ns/op           7.260 block_size(mb)           38.00 number_of_transactions      60412833 process_proposal_time(ns)      44369300 total_gas_used       200406 transactions_size(byte)

50_transactions_of_300000_bytes-16          1000000000           0.05762 ns/op           7.161 block_size(mb)           25.00 number_of_transactions      57616250 process_proposal_time(ns)      11202150 total_gas_used       300406 transactions_size(byte)

50_transactions_of_400000_bytes-16          1000000000           0.05062 ns/op           7.254 block_size(mb)           19.00 number_of_transactions      50619375 process_proposal_time(ns)      11202150 total_gas_used       400406 transactions_size(byte)

30_transactions_of_500000_bytes-16          1000000000           0.06729 ns/op           7.157 block_size(mb)           15.00 number_of_transactions      67294042 process_proposal_time(ns)       4085490 total_gas_used       500406 transactions_size(byte)

10_transactions_of_1000000_bytes-16         1000000000           0.04874 ns/op           6.678 block_size(mb)            7.000 number_of_transactions     48740916 process_proposal_time(ns)        483230 total_gas_used      1000406 transactions_size(byte)

5_transactions_of_2000000_bytes-16          1000000000           0.05007 ns/op           5.723 block_size(mb)            3.000 number_of_transactions     50073333 process_proposal_time(ns)        131790 total_gas_used      2000406 transactions_size(byte)

3_transactions_of_3000000_bytes-16          1000000000           0.04831 ns/op           5.723 block_size(mb)            2.000 number_of_transactions     48314167 process_proposal_time(ns)         52716 total_gas_used      3000409 transactions_size(byte)

3_transactions_of_4000000_bytes-16          1000000000           0.06277 ns/op           3.815 block_size(mb)            1.000 number_of_transactions     62765083 process_proposal_time(ns)         52716 total_gas_used      4000409 transactions_size(byte)

2_transactions_of_5000000_bytes-16          1000000000           0.03646 ns/op           4.769 block_size(mb)            1.000 number_of_transactions     36457375 process_proposal_time(ns)         26358 total_gas_used      5000409 transactions_size(byte)

2_transactions_of_6000000_bytes-16          1000000000           0.04704 ns/op           5.722 block_size(mb)            1.000 number_of_transactions     47042125 process_proposal_time(ns)         26358 total_gas_used      6000409 transactions_size(byte)

cc @evan-forbes @cmwaters

rach-id commented 1 month ago

Same for check tx, I added BenchmarkCheckTx_PFB_Multi which benchmarks check tx using different blob sizes. The results:

300_bytes-16                6654        175511 ns/op         61356 gas_used        703.0 transaction_size(byte)

500_bytes-16                6379        179029 ns/op         61356 gas_used        903.0 transaction_size(byte)

1000_bytes-16               6408        181527 ns/op         61356 gas_used       1403 transaction_size(byte)

5000_bytes-16               5929        201542 ns/op         61356 gas_used       5403 transaction_size(byte)

10000_bytes-16              5359        228696 ns/op         61356 gas_used      10403 transaction_size(byte)

50000_bytes-16              3518        332609 ns/op         61366 gas_used      50406 transaction_size(byte)

100000_bytes-16             2605        446551 ns/op         61366 gas_used     100406 transaction_size(byte)

200000_bytes-16             1786        666961 ns/op         61366 gas_used     200406 transaction_size(byte)

300000_bytes-16             1372        866706 ns/op         61366 gas_used     300406 transaction_size(byte)

400000_bytes-16             1076       1093213 ns/op         61366 gas_used     400406 transaction_size(byte)

500000_bytes-16              922       1289988 ns/op         61366 gas_used     500406 transaction_size(byte)

1000000_bytes-16             514       2329951 ns/op         61366 gas_used    1000406 transaction_size(byte)

2000000_bytes-16             270       4477223 ns/op         61366 gas_used    2000406 transaction_size(byte)

3000000_bytes-16             184       6467632 ns/op         61376 gas_used    3000409 transaction_size(byte)

4000000_bytes-16             140       8446076 ns/op         61376 gas_used    4000409 transaction_size(byte)

5000000_bytes-16             100      10610712 ns/op         61376 gas_used    5000409 transaction_size(byte)

6000000_bytes-16              88      12589757 ns/op         61376 gas_used    6000409 transaction_size(byte)

rach-id commented 1 month ago

Same for deliver tx, I added BenchmarkDeliverTx_PFB_Multi which benchmarks deliver tx using different blob sizes. The results:

300_bytes-16              126498          8503 ns/op    1111474019 gas_used        703.0 transaction_size(byte)

500_bytes-16              122810          9178 ns/op    1079071251 gas_used        903.0 transaction_size(byte)

1000_bytes-16             118024          9850 ns/op    1037021455 gas_used       1403 transaction_size(byte)

5000_bytes-16             101962         14902 ns/op     895900723 gas_used       5403 transaction_size(byte)

10000_bytes-16             75381         16963 ns/op     662360057 gas_used      10403 transaction_size(byte)

50000_bytes-16             36753         30369 ns/op     322974449 gas_used      50406 transaction_size(byte)

100000_bytes-16            23551         50086 ns/op     206981677 gas_used     100406 transaction_size(byte)

200000_bytes-16            13953         89059 ns/op     122653649 gas_used     200406 transaction_size(byte)

300000_bytes-16             8750        127090 ns/op      76940091 gas_used     300406 transaction_size(byte)

400000_bytes-16             7314        156464 ns/op      64323395 gas_used     400406 transaction_size(byte)

500000_bytes-16             6310        192877 ns/op      55502251 gas_used     500406 transaction_size(byte)

1000000_bytes-16            3226        362805 ns/op      28406227 gas_used    1000406 transaction_size(byte)

2000000_bytes-16            1641        705051 ns/op      14480417 gas_used    2000406 transaction_size(byte)

3000000_bytes-16            1129       1030048 ns/op       9981985 gas_used    3000409 transaction_size(byte)

4000000_bytes-16             880       1381749 ns/op       7794271 gas_used    4000409 transaction_size(byte)

5000000_bytes-16             696       1733789 ns/op       6177647 gas_used    5000409 transaction_size(byte)

6000000_bytes-16             548       2072252 ns/op       4877319 gas_used    6000409 transaction_size(byte)

rach-id commented 1 month ago

I will working on the remaining benchmarks and share the results once I have them

rach-id commented 1 month ago

If we're going to limit the gas in a block can you also work out the total gas of those two respective blocks I.e. (all MsgSend and all 4KB PFBs) 🙏

For an 8mb block full of msg send:

BenchmarkPrepareProposal_MsgSend_8MB-16            1    6119119000 ns/op             7.408 block_size(mb)        31645 number_of_transactions   2288906265 total_gas_used

For the PFBs, the total gas is in the previous comments.

rach-id commented 1 month ago

Can you also measure how long it takes to have 8MB worth of send msgs via DeliverTx - or can I just go 31645 * 53807 ns/op?

BenchmarkDeliverTx_MsgSend_8MB-16              1    2430262231 ns/op    1892513374 total_gas_used

rach-id commented 1 month ago

IBC update clientcheckTx benchmark with multiple signatures number in BenchmarkIBC_CheckTx_Update_Client_Multi:

number_of_validators:_2-16             16861         72076 ns/op         57713 gas_used          2.000 number_of_validators          1.000 number_of_verified_signatures          1399 transaction_size(byte)

number_of_validators:_10-16            12540         92223 ns/op         76753 gas_used         10.00 number_of_validators           6.000 number_of_verified_signatures          3303 transaction_size(byte)

number_of_validators:_25-16             9147        126315 ns/op        112473 gas_used         25.00 number_of_validators          16.00 number_of_verified_signatures       6875 transaction_size(byte)

number_of_validators:_50-16             6448        181547 ns/op        171973 gas_used         50.00 number_of_validators          33.00 number_of_verified_signatures      12825 transaction_size(byte)

number_of_validators:_75-16             4620        236575 ns/op        231523 gas_used         75.00 number_of_validators          50.00 number_of_verified_signatures      18780 transaction_size(byte)

number_of_validators:_100-16            3576        283364 ns/op        291023 gas_used        100.0 number_of_validators           66.00 number_of_verified_signatures      24730 transaction_size(byte)

number_of_validators:_125-16            3607        333925 ns/op        350523 gas_used        125.0 number_of_validators           83.00 number_of_verified_signatures      30680 transaction_size(byte)

number_of_validators:_150-16            3114        402968 ns/op        410023 gas_used        150.0 number_of_validators          100.0 number_of_verified_signatures       36630 transaction_size(byte)

number_of_validators:_175-16            2588        433434 ns/op        467783 gas_used        175.0 number_of_validators          116.0 number_of_verified_signatures       42406 transaction_size(byte)

number_of_validators:_200-16            2174        477910 ns/op        529043 gas_used        200.0 number_of_validators          133.0 number_of_verified_signatures       48532 transaction_size(byte)

number_of_validators:_225-16            2280        527478 ns/op        588543 gas_used        225.0 number_of_validators          150.0 number_of_verified_signatures       54482 transaction_size(byte)

number_of_validators:_250-16            1994        577420 ns/op        648063 gas_used        250.0 number_of_validators          166.0 number_of_verified_signatures       60434 transaction_size(byte)

number_of_validators:_300-16            1794        700452 ns/op        764053 gas_used        300.0 number_of_validators          200.0 number_of_verified_signatures       72033 transaction_size(byte)

number_of_validators:_400-16            1350        873867 ns/op       1005063 gas_used        400.0 number_of_validators          266.0 number_of_verified_signatures       96134 transaction_size(byte)

number_of_validators:_500-16            1023       1080929 ns/op       1243063 gas_used        500.0 number_of_validators          333.0 number_of_verified_signatures      119934 transaction_size(byte)

rach-id commented 1 month ago

IBC client update for PrepareProposal with multiple signatures numbers in BenchmarkIBC_PrepareProposal_Update_Client_Multi:

number_of_validators:_2-16                 1    2490856208 ns/op             7.457 block_size(mb)         5586 number_of_transactions            2.000 number_of_validators          1.000 number_of_verified_signatures    2490856208 prepare_proposal_time(ns)    1048626655 total_gas_used         1466 transactions_size(byte)

number_of_validators:_10-16                1    2602031959 ns/op             7.461 block_size(mb)         2374 number_of_transactions           10.00 number_of_validators           6.000 number_of_verified_signatures    2602031959 prepare_proposal_time(ns)    1333026655 total_gas_used         3362 transactions_size(byte)

number_of_validators:_25-16                1    2824113917 ns/op             7.462 block_size(mb)         1142 number_of_transactions           25.00 number_of_validators          16.00 number_of_verified_signatures 2824113917 prepare_proposal_time(ns)    1866576655 total_gas_used         6919 transactions_size(byte)

number_of_validators:_50-16                1    3233678584 ns/op             7.462 block_size(mb)          610.0 number_of_transactions         50.00 number_of_validators          33.00 number_of_verified_signatures 3233678584 prepare_proposal_time(ns)    2762976655 total_gas_used        12895 transactions_size(byte)

number_of_validators:_75-16                1    3661519083 ns/op             7.452 block_size(mb)          416.0 number_of_transactions         75.00 number_of_validators          50.00 number_of_verified_signatures 3661519083 prepare_proposal_time(ns)    3656226655 total_gas_used        18850 transactions_size(byte)

number_of_validators:_100-16               1    4005650209 ns/op             7.446 block_size(mb)          317.0 number_of_transactions        100.0 number_of_validators           66.00 number_of_verified_signatures 4005650209 prepare_proposal_time(ns)    4533576655 total_gas_used        24699 transactions_size(byte)

number_of_validators:_125-16               1    4371861042 ns/op             7.462 block_size(mb)          255.0 number_of_transactions        125.0 number_of_validators           83.00 number_of_verified_signatures 4371861042 prepare_proposal_time(ns)    5441226655 total_gas_used        30750 transactions_size(byte)

number_of_validators:_150-16               1    4826666125 ns/op             7.441 block_size(mb)          213.0 number_of_transactions        150.0 number_of_validators          100.0 number_of_verified_signatures  4826666125 prepare_proposal_time(ns)    6333726655 total_gas_used        36700 transactions_size(byte)

number_of_validators:_175-16               1    5010772208 ns/op             7.432 block_size(mb)          183.0 number_of_transactions        175.0 number_of_validators          116.0 number_of_verified_signatures  5010772208 prepare_proposal_time(ns)    7226526655 total_gas_used        42652 transactions_size(byte)

number_of_validators:_200-16               1    5300142167 ns/op             7.452 block_size(mb)          161.0 number_of_transactions        200.0 number_of_validators          133.0 number_of_verified_signatures  5300142167 prepare_proposal_time(ns)    8119026655 total_gas_used        48602 transactions_size(byte)

number_of_validators:_225-16               1    5631354084 ns/op             7.430 block_size(mb)          143.0 number_of_transactions        225.0 number_of_validators          150.0 number_of_verified_signatures  5631354084 prepare_proposal_time(ns)    9011526655 total_gas_used        54552 transactions_size(byte)

number_of_validators:_250-16               1    5931985458 ns/op             7.435 block_size(mb)          129.0 number_of_transactions        250.0 number_of_validators          166.0 number_of_verified_signatures  5931985458 prepare_proposal_time(ns)    9904326655 total_gas_used        60504 transactions_size(byte)

number_of_validators:_300-16               1    6663298708 ns/op             7.450 block_size(mb)          108.0 number_of_transactions        300.0 number_of_validators          200.0 number_of_verified_signatures  6663298708 prepare_proposal_time(ns)    11689326655 total_gas_used       72404 transactions_size(byte)

number_of_validators:_400-16               1    7987321625 ns/op             7.395 block_size(mb)           81.00 number_of_transactions           400.0 number_of_validators          266.0 number_of_verified_signatures  7987321625 prepare_proposal_time(ns)    15199176655 total_gas_used       95803 transactions_size(byte)

number_of_validators:_500-16               1    9172661459 ns/op             7.404 block_size(mb)           65.00 number_of_transactions           500.0 number_of_validators          333.0 number_of_verified_signatures  9172661459 prepare_proposal_time(ns)    18755826655 total_gas_used      119514 transactions_size(byte)

rach-id commented 1 month ago

IBC client update for ProcessProposal with multiple signatures numbers in BenchmarkIBC_ProcessProposal_Update_Client_Multi:

number_of_validators:_2-16                 2     512267416 ns/op             7.457 block_size(mb)         5586 number_of_transactions            2.000 number_of_validators          1.000 number_of_verified_signatures    1024534833 process_proposal_time(ns)     419431655 total_gas_used         1466 transactions_size(byte)

number_of_validators:_10-16         1000000000           0.4794 ns/op            7.464 block_size(mb)         2367 number_of_transactions           10.00 number_of_validators           6.000 number_of_verified_signatures     479357459 process_proposal_time(ns)     266926655 total_gas_used         3373 transactions_size(byte)

number_of_validators:_25-16         1000000000           0.2656 ns/op            7.462 block_size(mb)         1142 number_of_transactions           25.00 number_of_validators          16.00 number_of_verified_signatures  265567709 process_proposal_time(ns)     248871655 total_gas_used         6919 transactions_size(byte)

number_of_validators:_50-16         1000000000           0.1676 ns/op            7.462 block_size(mb)          610.0 number_of_transactions         50.00 number_of_validators          33.00 number_of_verified_signatures  167567000 process_proposal_time(ns)     184196655 total_gas_used        12895 transactions_size(byte)

number_of_validators:_75-16         1000000000           0.1518 ns/op            7.452 block_size(mb)          416.0 number_of_transactions         75.00 number_of_validators          50.00 number_of_verified_signatures  151821292 process_proposal_time(ns)     121879155 total_gas_used        18850 transactions_size(byte)

number_of_validators:_100-16        1000000000           0.1301 ns/op            7.453 block_size(mb)          316.0 number_of_transactions        100.0 number_of_validators           66.00 number_of_verified_signatures  130123833 process_proposal_time(ns)     151629155 total_gas_used        24800 transactions_size(byte)

number_of_validators:_125-16        1000000000           0.1086 ns/op            7.462 block_size(mb)          255.0 number_of_transactions        125.0 number_of_validators           83.00 number_of_verified_signatures  108590667 process_proposal_time(ns)     181379155 total_gas_used        30750 transactions_size(byte)

number_of_validators:_150-16        1000000000           0.1141 ns/op            7.441 block_size(mb)          213.0 number_of_transactions        150.0 number_of_validators          100.0 number_of_verified_signatures   114139959 process_proposal_time(ns)     211129155 total_gas_used        36700 transactions_size(byte)

number_of_validators:_175-16        1000000000           0.1080 ns/op            7.442 block_size(mb)          184.0 number_of_transactions        175.0 number_of_validators          116.0 number_of_verified_signatures   107993709 process_proposal_time(ns)     240009155 total_gas_used        42476 transactions_size(byte)

number_of_validators:_200-16        1000000000           0.1017 ns/op            7.452 block_size(mb)          161.0 number_of_transactions        200.0 number_of_validators          133.0 number_of_verified_signatures   101731250 process_proposal_time(ns)     270639155 total_gas_used        48602 transactions_size(byte)

number_of_validators:_225-16        1000000000           0.09454 ns/op           7.430 block_size(mb)          143.0 number_of_transactions        225.0 number_of_validators          150.0 number_of_verified_signatures    94540125 process_proposal_time(ns)     300389155 total_gas_used        54552 transactions_size(byte)

number_of_validators:_250-16        1000000000           0.08676 ns/op           7.462 block_size(mb)          130.0 number_of_transactions        250.0 number_of_validators          166.0 number_of_verified_signatures    86761584 process_proposal_time(ns)     328894155 total_gas_used        60253 transactions_size(byte)

number_of_validators:_300-16        1000000000           0.08495 ns/op           7.450 block_size(mb)          108.0 number_of_transactions        300.0 number_of_validators          200.0 number_of_verified_signatures    84953416 process_proposal_time(ns)     389649155 total_gas_used        72404 transactions_size(byte)

number_of_validators:_400-16        1000000000           0.08527 ns/op           7.426 block_size(mb)           81.00 number_of_transactions           400.0 number_of_validators          266.0 number_of_verified_signatures    85268167 process_proposal_time(ns)     508649155 total_gas_used        96204 transactions_size(byte)

number_of_validators:_500-16        1000000000           0.08442 ns/op           7.435 block_size(mb)           65.00 number_of_transactions           500.0 number_of_validators          333.0 number_of_verified_signatures    84421292 process_proposal_time(ns)     627649155 total_gas_used       120004 transactions_size(byte)

rach-id commented 1 month ago

IBC update clientdeliverTx benchmark with multiple signatures number in BenchmarkIBC_DeliverTx_Update_Client_Multi:

number_of_validators:_2-16          1000000000           0.0004954 ns/op         59956 gas_used          2.000 number_of_validators          1.000 number_of_verified_signatures          1469 transaction_size(byte)

number_of_validators:_10-16         1000000000           0.0005741 ns/op         78996 gas_used         10.00 number_of_validators           6.000 number_of_verified_signatures          3373 transaction_size(byte)

number_of_validators:_25-16         1000000000           0.0003831 ns/op        114716 gas_used         25.00 number_of_validators          16.00 number_of_verified_signatures       6945 transaction_size(byte)

number_of_validators:_50-16         1000000000           0.0002740 ns/op        174216 gas_used         50.00 number_of_validators          33.00 number_of_verified_signatures      12895 transaction_size(byte)

number_of_validators:_75-16         1000000000           0.0004448 ns/op        233006 gas_used         75.00 number_of_validators          50.00 number_of_verified_signatures      18774 transaction_size(byte)

number_of_validators:_100-16        1000000000           0.0008452 ns/op        292256 gas_used        100.0 number_of_validators           66.00 number_of_verified_signatures      24699 transaction_size(byte)

number_of_validators:_125-16        1000000000           0.0007626 ns/op        352766 gas_used        125.0 number_of_validators           83.00 number_of_verified_signatures      30750 transaction_size(byte)

number_of_validators:_150-16        1000000000           0.0005651 ns/op        412266 gas_used        150.0 number_of_validators          100.0 number_of_verified_signatures       36700 transaction_size(byte)

number_of_validators:_175-16        1000000000           0.0007676 ns/op        471786 gas_used        175.0 number_of_validators          116.0 number_of_verified_signatures       42652 transaction_size(byte)

number_of_validators:_200-16        1000000000           0.0008135 ns/op        531286 gas_used        200.0 number_of_validators          133.0 number_of_verified_signatures       48602 transaction_size(byte)

number_of_validators:_225-16        1000000000           0.001136 ns/op     588526 gas_used        225.0 number_of_validators          150.0 number_of_verified_signatures       54326 transaction_size(byte)

number_of_validators:_250-16        1000000000           0.001299 ns/op     650306 gas_used        250.0 number_of_validators          166.0 number_of_verified_signatures       60504 transaction_size(byte)

number_of_validators:_300-16        1000000000           0.001327 ns/op     769306 gas_used        300.0 number_of_validators          200.0 number_of_verified_signatures       72404 transaction_size(byte)

number_of_validators:_400-16        1000000000           0.001281 ns/op    1003296 gas_used        400.0 number_of_validators          266.0 number_of_verified_signatures       95803 transaction_size(byte)

number_of_validators:_500-16        1000000000           0.001369 ns/op    1245306 gas_used        500.0 number_of_validators          333.0 number_of_verified_signatures      120004 transaction_size(byte)

rach-id commented 1 month ago

This is ready for review :D :D

rach-id commented 1 month ago

@coderabbitai pause