search

AztecProtocol / aztec-packages

Apache License 2.0
155 stars 157 forks source link

feat: Full test skips public simulation #7186

Closed codygunton closed 1 week ago

codygunton commented 1 week ago

The e2e prover test skips public simulation. Change suggested by @sirasistant

AztecBot commented 1 week ago

Benchmark results

Metrics with a significant change:

Detailed results All benchmarks are run on txs on the `Benchmarking` contract on the repository. Each tx consists of a batch call to `create_note` and `increment_balance`, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write. This benchmark source data is available in JSON format on S3 [here](https://aztec-ci-artifacts.s3.us-east-2.amazonaws.com/benchmarks-v1/pulls/7186.json). ### Proof generation Each column represents the number of threads used in proof generation. | Metric | 1 threads | 4 threads | 16 threads | 32 threads | 64 threads | | - | - | - | - | - | - | proof_construction_time_sha256_30_ms | 11,548 (-2%) | 3,136 | 1,395 (-4%) | 1,529 (-8%) | 1,466 (-6%) | proof_construction_time_sha256_100_ms | 43,462 (-3%) | 11,588 (-1%) | 5,383 (-1%) | 5,355 (-1%) | 5,351 | proof_construction_time_poseidon_hash_ms | 76.0 (-3%) | 33.0 (-3%) | 33.0 (-3%) | 57.0 | 84.0 (-3%) | proof_construction_time_poseidon_hash_30_ms | 1,467 (-3%) | 402 (-3%) | 192 (-4%) | 221 | 256 (-4%) | proof_construction_time_poseidon_hash_100_ms | 5,550 (-3%) | 1,525 (-3%) | 697 (-4%) | 737 (-6%) | 771 (-2%) | ### L2 block published to L1 Each column represents the number of txs on an L2 block published to L1. | Metric | 4 txs | 8 txs | 16 txs | | - | - | - | - | l1_rollup_calldata_size_in_bytes | 1,412 | 1,412 | 1,412 | l1_rollup_calldata_gas | 9,476 | 9,458 | 9,440 | l1_rollup_execution_gas | 610,297 | 610,279 | 610,261 | l2_block_processing_time_in_ms | 763 (+1%) | 1,422 | 2,722 (-1%) | l2_block_building_time_in_ms | 25,295 (-1%) | 50,485 (-3%) | 99,629 | l2_block_rollup_simulation_time_in_ms | 25,202 (-1%) | 50,184 (-3%) | 99,276 | l2_block_public_tx_process_time_in_ms | 21,650 (-1%) | 46,466 (-3%) | 95,591 | ### L2 chain processing Each column represents the number of blocks on the L2 chain where each block has 8 txs. | Metric | 3 blocks | 5 blocks | | - | - | - | node_history_sync_time_in_ms | 7,091 (+1%) | 9,897 | node_database_size_in_bytes | 12,157,008 | 16,097,360 | pxe_database_size_in_bytes | 16,254 | 26,813 | ### Circuits stats Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks. | Circuit | simulation_time_in_ms | witness_generation_time_in_ms | proving_time_in_ms | input_size_in_bytes | output_size_in_bytes | proof_size_in_bytes | num_public_inputs | size_in_gates | | - | - | - | - | - | - | - | - | - | private-kernel-init | 129 (-1%) | 459 (-5%) | 13,119 (+4%) | 20,634 | 67,190 | 92,352 | 2,819 | 524,288 | private-kernel-inner | 391 (-1%) | :warning: 1,233 (**+30%**) | 48,227 (-4%) | 94,902 | 67,190 | 92,352 | 2,819 | 2,097,152 | private-kernel-tail | 369 (-1%) | 1,802 (+2%) | 52,337 (+5%) | 99,121 | 71,733 | 14,912 | 399 | 2,097,152 | base-parity | 6.16 (-1%) | 1,876 | 2,740 (+1%) | 128 | 64.0 | 2,208 | 2.00 | 131,072 | root-parity | 48.4 (-1%) | 71.5 (-5%) | 42,216 (+5%) | 27,100 | 64.0 | 2,720 | 18.0 | 2,097,152 | base-rollup | 7,823 (-2%) | 4,900 (-1%) | 84,674 (-1%) | 170,330 | 756 | 3,648 | 47.0 | 4,194,304 | root-rollup | 109 | 86.8 (+1%) | 22,950 | 25,309 | 620 | 3,456 | 41.0 | 1,048,576 | public-kernel-setup | 709 | 3,618 | 45,636 (+2%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-app-logic | 606 (-2%) | 4,705 (+2%) | 46,803 (+4%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-tail | 1,394 (-2%) | 37,856 (-4%) | 190,489 | 511,910 | 10,014 | 14,912 | 399 | 8,388,608 | private-kernel-reset-small | 589 (-2%) | 2,023 (-2%) | 46,219 (+1%) | 123,313 | 67,190 | 92,352 | 2,819 | 2,097,152 | public-kernel-teardown | 613 | 4,595 (-3%) | 45,188 (+1%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | merge-rollup | 28.7 | N/A | N/A | 16,542 | 756 | N/A | N/A | N/A | private-kernel-tail-to-public | N/A | :warning: 8,035 (**+17%**) | 97,820 (+2%) | N/A | N/A | 125,344 | 3,850 | 4,194,304 | Stats on running time collected for app circuits | Function | input_size_in_bytes | output_size_in_bytes | witness_generation_time_in_ms | proof_size_in_bytes | proving_time_in_ms | size_in_gates | num_public_inputs | | - | - | - | - | - | - | - | - | ContractClassRegisterer:register | 1,344 | 9,944 | 422 | N/A | N/A | N/A | N/A | ContractInstanceDeployer:deploy | 1,408 | 9,944 | 39.6 | N/A | N/A | N/A | N/A | MultiCallEntrypoint:entrypoint | 1,920 | 9,944 | 1,782 | N/A | N/A | N/A | N/A | GasToken:deploy | 1,376 | 9,944 | 965 (+1%) | N/A | N/A | N/A | N/A | SchnorrAccount:constructor | 1,312 | 9,944 | 1,404 | N/A | N/A | N/A | N/A | SchnorrAccount:entrypoint | 2,304 | 9,944 | 2,813 | 16,768 | 54,732 (-1%) | 2,097,152 | 457 | Token:privately_mint_private_note | 1,280 | 9,944 | 1,611 (-1%) | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_public | 1,344 | 9,944 | 357 (+3%) | 16,768 | 11,296 | 524,288 | 457 | Token:transfer | 1,312 | 9,944 | 4,478 (+1%) | 16,768 | 45,908 (-6%) | 2,097,152 | 457 | AuthRegistry:set_authorized (avm) | 21,043 | N/A | N/A | 87,200 | 1,372 (+3%) | N/A | N/A | FPC:prepare_fee (avm) | 28,841 | N/A | N/A | 88,032 | 5,731 (+3%) | N/A | N/A | Token:transfer_public (avm) | 44,971 | N/A | N/A | 87,865 | 4,696 (+1%) | N/A | N/A | AuthRegistry:consume (avm) | 34,973 | N/A | N/A | 87,616 | 2,981 | N/A | N/A | FPC:pay_refund (avm) | 33,573 | N/A | N/A | 88,448 | 11,137 (-2%) | N/A | N/A | Benchmarking:create_note | 1,344 | 9,944 | 1,399 | N/A | N/A | N/A | N/A | SchnorrAccount:verify_private_authwit | 1,280 | 9,944 | 73.1 (-5%) | N/A | N/A | N/A | N/A | Token:unshield | 1,376 | 9,944 | 3,701 (+1%) | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_private | 1,376 | 9,944 | 4,631 | N/A | N/A | N/A | N/A | ### AVM Simulation Time to simulate various public functions in the AVM. | Function | time_ms | bytecode_size_in_bytes | | - | - | - | GasToken:_increase_public_balance | 68.5 | 13,873 | GasToken:set_portal | 16.7 (-1%) | 3,495 | Token:constructor | 92.6 (+1%) | 24,207 | FPC:constructor | 61.6 (-2%) | 13,893 | GasToken:mint_public | 51.0 | 10,241 | Token:mint_public | :warning: 600 (**+855%**) | 19,216 | Token:assert_minter_and_mint | :warning: 71.4 (**-65%**) | 13,034 | AuthRegistry:set_authorized | 31.4 (+5%) | 7,869 | FPC:prepare_fee | 136 (-29%) | 15,187 | Token:transfer_public | 48.7 (-26%) | 31,425 | FPC:pay_refund | 138 (+22%) | 20,080 | Benchmarking:increment_balance | 2,660 (-2%) | 15,465 | Token:_increase_public_balance | 55.4 (-1%) | 15,089 | FPC:pay_refund_with_shielded_rebate | 150 (+6%) | 21,167 | ### Public DB Access Time to access various public DBs. | Function | time_ms | | - | - | get-nullifier-index | 0.152 (-1%) | ### Tree insertion stats The duration to insert a fixed batch of leaves into each tree type. | Metric | 1 leaves | 16 leaves | 64 leaves | 128 leaves | 256 leaves | 512 leaves | 1024 leaves | | - | - | - | - | - | - | - | - | batch_insert_into_append_only_tree_16_depth_ms | 10.3 (-5%) | 16.7 (-4%) | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_16_depth_hash_count | 16.8 | 31.7 | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_16_depth_hash_ms | 0.596 (-5%) | 0.512 (-4%) | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_32_depth_ms | N/A | N/A | 48.4 | 75.9 (-12%) | 132 (+1%) | 244 (-1%) | 470 (-10%) | batch_insert_into_append_only_tree_32_depth_hash_count | N/A | N/A | 95.9 | 159 | 287 | 543 | 1,055 | batch_insert_into_append_only_tree_32_depth_hash_ms | N/A | N/A | 0.495 | 0.467 (-12%) | 0.452 | 0.442 (-1%) | 0.439 (-10%) | batch_insert_into_indexed_tree_20_depth_ms | N/A | N/A | 59.4 | 111 (-11%) | 181 (-2%) | 351 (-1%) | 693 | batch_insert_into_indexed_tree_20_depth_hash_count | N/A | N/A | 109 | 207 | 355 | 691 | 1,363 | batch_insert_into_indexed_tree_20_depth_hash_ms | N/A | N/A | 0.503 | 0.500 (-11%) | 0.480 (-2%) | 0.476 (-1%) | 0.475 | batch_insert_into_indexed_tree_40_depth_ms | N/A | N/A | 72.6 (-2%) | N/A | N/A | N/A | N/A | batch_insert_into_indexed_tree_40_depth_hash_count | N/A | N/A | 133 | N/A | N/A | N/A | N/A | batch_insert_into_indexed_tree_40_depth_hash_ms | N/A | N/A | 0.517 (-2%) | N/A | N/A | N/A | N/A | ### Miscellaneous Transaction sizes based on how many contract classes are registered in the tx. | Metric | 0 registered classes | 1 registered classes | | - | - | - | tx_size_in_bytes | 85,672 | 670,983 | Transaction size based on fee payment method | Metric | | | - | |