search

AztecProtocol / aztec-packages

Apache License 2.0
180 stars 184 forks source link

chore: misc cleanup in simulator #7203

Closed dbanks12 closed 3 months ago

dbanks12 commented 3 months ago

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @dbanks12 and the rest of your teammates on Graphite Graphite

AztecBot commented 3 months ago

Benchmark results

Metrics with a significant change:

Detailed results All benchmarks are run on txs on the `Benchmarking` contract on the repository. Each tx consists of a batch call to `create_note` and `increment_balance`, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write. This benchmark source data is available in JSON format on S3 [here](https://aztec-ci-artifacts.s3.us-east-2.amazonaws.com/benchmarks-v1/pulls/7203.json). ### Proof generation Each column represents the number of threads used in proof generation. | Metric | 1 threads | 4 threads | 16 threads | 32 threads | 64 threads | | - | - | - | - | - | - | proof_construction_time_sha256_30_ms | 11,383 (-1%) | 3,057 (-1%) | :warning: 1,694 (**+22%**) | 1,655 | 1,537 (+5%) | proof_construction_time_sha256_100_ms | 43,852 (-1%) | 11,732 (-1%) | 5,473 (+1%) | 5,384 | 5,355 | proof_construction_time_poseidon_hash_ms | 78.0 (-1%) | 34.0 | 34.0 | 58.0 (+2%) | 89.0 (+2%) | proof_construction_time_poseidon_hash_30_ms | 1,516 (-1%) | 414 (-1%) | 200 | 229 (+5%) | 266 | proof_construction_time_poseidon_hash_100_ms | 5,743 (-1%) | 1,562 (-1%) | 722 (+1%) | 783 | 800 (+1%) | ### L2 block published to L1 Each column represents the number of txs on an L2 block published to L1. | Metric | 4 txs | 8 txs | 16 txs | | - | - | - | - | l1_rollup_calldata_size_in_bytes | 1,412 | 1,412 | 1,412 | l1_rollup_calldata_gas | 9,476 | 9,462 | 9,476 | l1_rollup_execution_gas | 610,297 | 610,283 | 610,297 | l2_block_processing_time_in_ms | 748 | 1,415 (+1%) | 2,679 (-1%) | l2_block_building_time_in_ms | 25,529 (+2%) | 50,834 (+1%) | 99,909 (+1%) | l2_block_rollup_simulation_time_in_ms | 25,529 (+2%) | 50,834 (+1%) | 99,909 (+1%) | l2_block_public_tx_process_time_in_ms | 21,940 (+2%) | 47,056 (+1%) | 96,176 (+1%) | ### L2 chain processing Each column represents the number of blocks on the L2 chain where each block has 8 txs. | Metric | 3 blocks | 5 blocks | | - | - | - | node_history_sync_time_in_ms | 7,075 (+1%) | 9,908 (+1%) | node_database_size_in_bytes | 12,152,912 | 16,093,264 | pxe_database_size_in_bytes | 16,254 | 26,813 | ### Circuits stats Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks. | Circuit | simulation_time_in_ms | witness_generation_time_in_ms | proving_time_in_ms | input_size_in_bytes | output_size_in_bytes | proof_size_in_bytes | num_public_inputs | size_in_gates | | - | - | - | - | - | - | - | - | - | private-kernel-init | 129 | 490 (+3%) | 11,937 (-7%) | 20,634 | 67,190 | 92,352 | 2,819 | 524,288 | private-kernel-inner | 393 (+1%) | 1,001 (-10%) | 44,432 (-11%) | 94,902 | 67,190 | 92,352 | 2,819 | 2,097,152 | private-kernel-tail | 371 (+1%) | 1,852 (+1%) | :warning: 45,624 (**-19%**) | 99,121 | 71,733 | 14,912 | 399 | 2,097,152 | base-parity | 6.27 (-1%) | 1,934 (+2%) | 2,536 (-9%) | 128 | 64.0 | 2,208 | 2.00 | 131,072 | root-parity | 49.9 (+2%) | 56.6 (-25%) | :warning: 34,072 (**-16%**) | 27,100 | 64.0 | 2,720 | 18.0 | 2,097,152 | base-rollup | 7,942 (+1%) | 4,985 (+4%) | 72,583 (-15%) | 170,330 | 756 | 3,648 | 47.0 | 4,194,304 | root-rollup | 112 (+3%) | :warning: 73.0 (**-35%**) | :warning: 19,511 (**-22%**) | 25,309 | 620 | 3,456 | 41.0 | 1,048,576 | public-kernel-setup | 724 (+1%) | 3,633 (-2%) | 38,654 (-14%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-app-logic | 613 (+1%) | 4,782 (-2%) | 39,809 (-15%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-tail | 1,420 (+1%) | 37,446 (+4%) | 166,197 (-15%) | 511,910 | 10,014 | 14,912 | 399 | 8,388,608 | private-kernel-reset-small | 594 (+1%) | 2,167 (+10%) | 41,021 (-14%) | 123,313 | 67,190 | 92,352 | 2,819 | 2,097,152 | public-kernel-teardown | 619 | 4,834 (+2%) | :warning: 40,055 (**-17%**) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | merge-rollup | 29.0 | N/A | N/A | 16,542 | 756 | N/A | N/A | N/A | private-kernel-tail-to-public | N/A | 7,245 (+6%) | :warning: 84,173 (**-19%**) | N/A | N/A | 125,344 | 3,850 | 4,194,304 | Stats on running time collected for app circuits | Function | input_size_in_bytes | output_size_in_bytes | witness_generation_time_in_ms | proof_size_in_bytes | proving_time_in_ms | size_in_gates | num_public_inputs | | - | - | - | - | - | - | - | - | ContractClassRegisterer:register | 1,344 | 9,944 | 428 (+2%) | N/A | N/A | N/A | N/A | ContractInstanceDeployer:deploy | 1,408 | 9,944 | 40.5 (+2%) | N/A | N/A | N/A | N/A | MultiCallEntrypoint:entrypoint | 1,920 | 9,944 | 1,804 (+1%) | N/A | N/A | N/A | N/A | GasToken:deploy | 1,376 | 9,944 | 1,011 (+3%) | N/A | N/A | N/A | N/A | SchnorrAccount:constructor | 1,312 | 9,944 | 1,439 (+2%) | N/A | N/A | N/A | N/A | SchnorrAccount:entrypoint | 2,304 | 9,944 | 2,844 (+1%) | 16,768 | 51,019 (-10%) | 2,097,152 | 457 | Token:privately_mint_private_note | 1,280 | 9,944 | 1,700 (+5%) | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_public | 1,344 | 9,944 | 377 (+5%) | 16,768 | 10,195 (-9%) | 524,288 | 457 | Token:transfer | 1,312 | 9,944 | 4,572 (+1%) | 16,768 | 41,118 (-15%) | 2,097,152 | 457 | AuthRegistry:set_authorized (avm) | 21,043 | N/A | N/A | 87,200 | :warning: 1,607 (**+18%**) | N/A | N/A | FPC:prepare_fee (avm) | 28,553 | N/A | N/A | 88,032 | :warning: 4,836 (**-17%**) | N/A | N/A | Token:transfer_public (avm) | 44,885 | N/A | N/A | 87,754 | 3,665 (-9%) | N/A | N/A | AuthRegistry:consume (avm) | 34,973 | N/A | N/A | 87,616 | 2,837 (-4%) | N/A | N/A | FPC:pay_refund (avm) | 33,413 | N/A | N/A | 88,448 | :warning: 9,083 (**-19%**) | N/A | N/A | Benchmarking:create_note | 1,344 | 9,944 | 1,402 (+1%) | N/A | N/A | N/A | N/A | SchnorrAccount:verify_private_authwit | 1,280 | 9,944 | 73.2 | N/A | N/A | N/A | N/A | Token:unshield | 1,376 | 9,944 | 3,704 | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_private | 1,376 | 9,944 | 4,633 (-1%) | N/A | N/A | N/A | N/A | ### AVM Simulation Time to simulate various public functions in the AVM. | Function | time_ms | bytecode_size_in_bytes | | - | - | - | GasToken:_increase_public_balance | 74.9 (+10%) | 13,873 | GasToken:set_portal | 16.9 | 3,495 | Token:constructor | 94.9 (+2%) | 24,207 | FPC:constructor | 65.9 | 13,893 | GasToken:mint_public | 54.6 (+4%) | 10,241 | Token:mint_public | :warning: 625 (**+891%**) | 19,216 | Token:assert_minter_and_mint | :warning: 76.4 (**-63%**) | 13,034 | AuthRegistry:set_authorized | 32.9 (+2%) | 7,869 | FPC:prepare_fee | 188 (-6%) | 15,187 | Token:transfer_public | 38.6 (+11%) | 31,425 | FPC:pay_refund | 127 (-2%) | 20,080 | Benchmarking:increment_balance | 2,691 (+1%) | 15,465 | Token:_increase_public_balance | 60.5 (+8%) | 15,089 | FPC:pay_refund_with_shielded_rebate | 156 (+5%) | 21,167 | ### Public DB Access Time to access various public DBs. | Function | time_ms | | - | - | get-nullifier-index | 0.155 (-1%) | ### Tree insertion stats The duration to insert a fixed batch of leaves into each tree type. | Metric | 1 leaves | 16 leaves | 64 leaves | 128 leaves | 256 leaves | 512 leaves | 1024 leaves | | - | - | - | - | - | - | - | - | batch_insert_into_append_only_tree_16_depth_ms | 10.4 (+1%) | 16.8 (+1%) | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_16_depth_hash_count | 16.8 | 31.7 | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_16_depth_hash_ms | 0.602 (+1%) | 0.518 (+1%) | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_32_depth_ms | N/A | N/A | 48.7 | 76.2 (+1%) | 132 (+1%) | 246 (+1%) | 469 | batch_insert_into_append_only_tree_32_depth_hash_count | N/A | N/A | 95.9 | 159 | 287 | 543 | 1,055 | batch_insert_into_append_only_tree_32_depth_hash_ms | N/A | N/A | 0.498 | 0.468 (+1%) | 0.452 (+1%) | 0.445 (+1%) | 0.438 | batch_insert_into_indexed_tree_20_depth_ms | N/A | N/A | 60.0 (+1%) | 112 | 182 | 354 (+1%) | 691 | batch_insert_into_indexed_tree_20_depth_hash_count | N/A | N/A | 109 | 207 | 355 | 691 | 1,363 | batch_insert_into_indexed_tree_20_depth_hash_ms | N/A | N/A | 0.508 (+1%) | 0.502 | 0.483 | 0.479 (+1%) | 0.474 | batch_insert_into_indexed_tree_40_depth_ms | N/A | N/A | 73.3 (+1%) | N/A | N/A | N/A | N/A | batch_insert_into_indexed_tree_40_depth_hash_count | N/A | N/A | 133 | N/A | N/A | N/A | N/A | batch_insert_into_indexed_tree_40_depth_hash_ms | N/A | N/A | 0.521 (+1%) | N/A | N/A | N/A | N/A | ### Miscellaneous Transaction sizes based on how many contract classes are registered in the tx. | Metric | 0 registered classes | 1 registered classes | | - | - | - | tx_size_in_bytes | 85,672 | 670,983 | Transaction size based on fee payment method | Metric | | | - | |
github-actions[bot] commented 3 months ago

Changes to circuit sizes

Generated at commit: 15a293b643ab8e4665e4a5016ee93138399838e1, compared to commit: b4f50a5bf03babd83c1f467d2792b50e334bf5a7

🧾 Summary (100% most significant diffs)

Program ACIR opcodes (+/-) % Circuit size (+/-) %
rollup_base +131 ❌ +0.04% +32,897 ❌ +1.07%
private_kernel_reset_small +244 ❌ +0.19% +235 ❌ +0.02%
private_kernel_reset_medium +244 ❌ +0.17% +235 ❌ +0.02%
private_kernel_reset_big +244 ❌ +0.14% +235 ❌ +0.01%
private_kernel_reset +244 ❌ +0.10% +235 ❌ +0.01%
public_kernel_tail +258 ❌ +0.02% +383 ❌ +0.01%
public_kernel_setup +5 ❌ +0.00% +5 ❌ +0.00%
private_kernel_init +1 ❌ +0.00% +1 ❌ +0.00%
private_kernel_tail_to_public +5 ❌ +0.00% +5 ❌ +0.00%
private_kernel_inner +1 ❌ +0.00% +1 ❌ +0.00%
public_kernel_app_logic +1 ❌ +0.00% +1 ❌ +0.00%
public_kernel_teardown +1 ❌ +0.00% +1 ❌ +0.00%
Full diff report 👇
| Program | ACIR opcodes (+/-) | % | Circuit size (+/-) | % | |:-|-:|-:|-:|-:| | **rollup_base** | 339,099 (+131) | **+0.04%** | 3,096,787 (+32,897) | **+1.07%** | | **private_kernel_reset_small** | 131,484 (+244) | **+0.19%** | 1,071,601 (+235) | **+0.02%** | | **private_kernel_reset_medium** | 147,281 (+244) | **+0.17%** | 1,282,036 (+235) | **+0.02%** | | **private_kernel_reset_big** | 178,873 (+244) | **+0.14%** | 1,702,904 (+235) | **+0.01%** | | **private_kernel_reset** | 242,060 (+244) | **+0.10%** | 2,544,643 (+235) | **+0.01%** | | **public_kernel_tail** | 1,216,847 (+258) | **+0.02%** | 5,427,095 (+383) | **+0.01%** | | **public_kernel_setup** | 280,960 (+5) | **+0.00%** | 1,802,698 (+5) | **+0.00%** | | **private_kernel_init** | 30,687 (+1) | **+0.00%** | 390,776 (+1) | **+0.00%** | | **private_kernel_tail_to_public** | 458,567 (+5) | **+0.00%** | 2,118,159 (+5) | **+0.00%** | | **private_kernel_inner** | 53,511 (+1) | **+0.00%** | 1,292,649 (+1) | **+0.00%** | | **public_kernel_app_logic** | 340,522 (+1) | **+0.00%** | 2,026,182 (+1) | **+0.00%** | | **public_kernel_teardown** | 340,720 (+1) | **+0.00%** | 2,026,421 (+1) | **+0.00%** |