issues
search
AztecProtocol
/
aztec-packages
Apache License 2.0
184
stars
184
forks
source link
fix: setup-action
#7268
Closed
spypsy
closed
3 months ago
AztecBot
commented
3 months ago
Benchmark results
Metrics with a significant change:
proof_construction_time_sha256_30_ms
(16): 1,437 (
-16%
)
avm_simulation_time_ms
(Token:mint_public): 87.2 (
-86%
)
avm_simulation_time_ms
(Token:assert_minter_and_mint): 217 (
+201%
)
avm_simulation_time_ms
(Token:transfer_public): 37.0 (
-39%
)
protocol_circuit_witness_generation_time_in_ms
(private-kernel-inner): 1,145 (
+22%
)
Detailed results
All benchmarks are run on txs on the `Benchmarking` contract on the repository. Each tx consists of a batch call to `create_note` and `increment_balance`, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write. This benchmark source data is available in JSON format on S3 [here](https://aztec-ci-artifacts.s3.us-east-2.amazonaws.com/benchmarks-v1/pulls/7268.json). ### Proof generation Each column represents the number of threads used in proof generation. | Metric | 1 threads | 4 threads | 16 threads | 32 threads | 64 threads | | - | - | - | - | - | - |
proof_construction_time_sha256_30_ms
| 11,712 (
-2%
) | 3,143 (
-2%
) | :warning: 1,437 (**
-16%
**) | 1,598 (
+10%
) | 1,591 (
+2%
) |
proof_construction_time_sha256_100_ms
| 44,251 | 11,760 (
-1%
) | 5,436 | 5,411 | 5,363 |
proof_construction_time_poseidon_hash_ms
| 77.0 (
-3%
) | 34.0 (
-26%
) | 34.0 (
-3%
) | 59.0 | 87.0 (
-1%
) |
proof_construction_time_poseidon_hash_30_ms
| 1,518 (
-1%
) | 416 (
-1%
) | 200 (
+1%
) | 226 (
-1%
) | 267 |
proof_construction_time_poseidon_hash_100_ms
| 5,762 (
-1%
) | 1,583 | 724 | 769 (
-3%
) | 798 (
+1%
) | ### L2 block published to L1 Each column represents the number of txs on an L2 block published to L1. | Metric | 4 txs | 8 txs | 16 txs | | - | - | - | - |
l1_rollup_calldata_size_in_bytes
| 1,412 | 1,412 | 1,412 |
l1_rollup_calldata_gas
| 9,464 | 9,464 | 9,464 |
l1_rollup_execution_gas
| 610,285 | 610,285 | 610,285 |
l2_block_processing_time_in_ms
| 762 | 1,409 (
-1%
) | 2,713 (
-1%
) |
l2_block_building_time_in_ms
| 25,231 (
-1%
) | 51,227 (
+1%
) | 99,188 (
-1%
) |
l2_block_rollup_simulation_time_in_ms
| 25,230 (
-1%
) | 51,226 (
+1%
) | 99,187 (
-1%
) |
l2_block_public_tx_process_time_in_ms
| 21,685 (
-1%
) | 47,453 (
+1%
) | 95,493 (
-1%
) | ### L2 chain processing Each column represents the number of blocks on the L2 chain where each block has 8 txs. | Metric | 3 blocks | 5 blocks | | - | - | - |
node_history_sync_time_in_ms
| 7,042 (
-1%
) | 9,846 (
-1%
) |
node_database_size_in_bytes
| 12,193,872 | 16,126,032 |
pxe_database_size_in_bytes
| 16,254 | 26,813 | ### Circuits stats Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks. | Circuit | simulation_time_in_ms | witness_generation_time_in_ms | proving_time_in_ms | input_size_in_bytes | output_size_in_bytes | proof_size_in_bytes | num_public_inputs | size_in_gates | | - | - | - | - | - | - | - | - | - | private-kernel-init | 129 (
-2%
) | 482 (
+1%
) | 13,161 (
+1%
) | 20,634 | 67,190 | 92,352 | 2,819 | 524,288 | private-kernel-inner | 392 | :warning: 1,145 (**
+22%
**) | 49,195 (
-1%
) | 94,902 | 67,190 | 92,352 | 2,819 | 2,097,152 | private-kernel-tail | 370 | 1,817 (
-1%
) | 51,716 (
+2%
) | 99,121 | 71,733 | 14,912 | 399 | 2,097,152 | base-parity | 6.15 | 1,870 (
-4%
) | 2,777 (
+2%
) | 128 | 64.0 | 2,208 | 2.00 | 131,072 | root-parity | 49.1 (
+1%
) | 77.2 (
+5%
) | 41,613 (
-4%
) | 27,100 | 64.0 | 2,720 | 18.0 | 2,097,152 | base-rollup | 7,933 | 4,920 | 83,812 (
-7%
) | 170,330 | 756 | 3,648 | 47.0 | 4,194,304 | root-rollup | 109 (
-1%
) | 91.2 (
+3%
) | 24,557 (
+11%
) | 25,309 | 620 | 3,456 | 41.0 | 1,048,576 | public-kernel-setup | 714 | 3,646 (
-2%
) | 45,633 (
+1%
) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-app-logic | 612 | 4,693 (
-2%
) | 43,996 (
-7%
) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-tail | 1,411 | 36,294 (
-4%
) | 194,219 (
-1%
) | 511,910 | 10,014 | 14,912 | 399 | 8,388,608 | private-kernel-reset-small | 593 | 2,171 (
+3%
) | 48,031 (
+2%
) | 123,313 | 67,190 | 92,352 | 2,819 | 2,097,152 | public-kernel-teardown | 612 | 4,678 | 46,623 (
-1%
) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | merge-rollup | 28.6 | N/A | N/A | 16,542 | 756 | N/A | N/A | N/A | private-kernel-tail-to-public | N/A | 7,333 (
-5%
) | 99,409 (
-1%
) | N/A | N/A | 125,344 | 3,850 | 4,194,304 | Stats on running time collected for app circuits | Function | input_size_in_bytes | output_size_in_bytes | witness_generation_time_in_ms | proof_size_in_bytes | proving_time_in_ms | size_in_gates | num_public_inputs | | - | - | - | - | - | - | - | - | ContractClassRegisterer:register | 1,344 | 9,944 | 420 (
-1%
) | N/A | N/A | N/A | N/A | ContractInstanceDeployer:deploy | 1,408 | 9,944 | 39.4 (
-1%
) | N/A | N/A | N/A | N/A | MultiCallEntrypoint:entrypoint | 1,920 | 9,944 | 1,766 (
-1%
) | N/A | N/A | N/A | N/A | GasToken:deploy | 1,376 | 9,944 | 975 (
-1%
) | N/A | N/A | N/A | N/A | SchnorrAccount:constructor | 1,312 | 9,944 | 1,400 (
-1%
) | N/A | N/A | N/A | N/A | SchnorrAccount:entrypoint | 2,304 | 9,944 | 2,814 | 16,768 | 56,685 (
+3%
) | 2,097,152 | 457 | Token:privately_mint_private_note | 1,280 | 9,944 | 1,592 (
-5%
) | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_public | 1,344 | 9,944 | 353 | 16,768 | 11,193 | 524,288 | 457 | Token:transfer | 1,312 | 9,944 | 4,467 (
-1%
) | 16,768 | 45,704 (
-5%
) | 2,097,152 | 457 | AuthRegistry:set_authorized (avm) | 21,043 | N/A | N/A | 87,200 | 1,371 (
+1%
) | N/A | N/A | FPC:prepare_fee (avm) | 28,521 | N/A | N/A | 88,032 | 5,719 (
-4%
) | N/A | N/A | AuthRegistry:consume (avm) | 34,973 | N/A | N/A | 87,616 | 2,986 (
-2%
) | N/A | N/A | Token:transfer_public (avm) | 44,849 | N/A | N/A | 87,616 | 2,983 (
-2%
) | N/A | N/A | FPC:pay_refund (avm) | 33,381 | N/A | N/A | 88,448 | 11,006 (
-4%
) | N/A | N/A | Benchmarking:create_note | 1,344 | 9,944 | 1,395 (
-1%
) | N/A | N/A | N/A | N/A | SchnorrAccount:verify_private_authwit | 1,280 | 9,944 | 73.1 | N/A | N/A | N/A | N/A | Token:unshield | 1,376 | 9,944 | 3,688 | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_private | 1,376 | 9,944 | 4,621 (
-1%
) | N/A | N/A | N/A | N/A | ### AVM Simulation Time to simulate various public functions in the AVM. | Function | time_ms | bytecode_size_in_bytes | | - | - | - | GasToken:_increase_public_balance | 70.5 (
-3%
) | 13,873 | GasToken:set_portal | 17.0 (
+1%
) | 3,495 | Token:constructor | 90.2 (
-2%
) | 24,207 | FPC:constructor | 63.3 (
-1%
) | 13,893 | GasToken:mint_public | 52.7 (
+3%
) | 10,241 | Token:mint_public | :warning: 87.2 (**
-86%
**) | 19,216 | Token:assert_minter_and_mint | :warning: 217 (**
+201%
**) | 13,034 | AuthRegistry:set_authorized | 33.5 (
+8%
) | 7,869 | FPC:prepare_fee | 188 (
+4%
) | 15,187 | Token:transfer_public | :warning: 37.0 (**
-39%
**) | 31,425 | FPC:pay_refund | 137 (
+10%
) | 20,080 | Benchmarking:increment_balance | 2,696 | 15,465 | Token:_increase_public_balance | 56.2 (
-1%
) | 15,089 | FPC:pay_refund_with_shielded_rebate | 145 (
-3%
) | 21,167 | ### Public DB Access Time to access various public DBs. | Function | time_ms | | - | - | get-nullifier-index | 0.176 (
+16%
) | ### Tree insertion stats The duration to insert a fixed batch of leaves into each tree type. | Metric | 1 leaves | 16 leaves | 64 leaves | 128 leaves | 256 leaves | 512 leaves | 1024 leaves | | - | - | - | - | - | - | - | - |
batch_insert_into_append_only_tree_16_depth_ms
| 10.3 | 16.7 (
-1%
) | N/A | N/A | N/A | N/A | N/A |
batch_insert_into_append_only_tree_16_depth_hash_count
| 16.8 | 31.7 | N/A | N/A | N/A | N/A | N/A |
batch_insert_into_append_only_tree_16_depth_hash_ms
| 0.597 | 0.513 (
-1%
) | N/A | N/A | N/A | N/A | N/A |
batch_insert_into_append_only_tree_32_depth_ms
| N/A | N/A | 48.5 | 76.4 | 132 (
+1%
) | 245 | 471 (
-1%
) |
batch_insert_into_append_only_tree_32_depth_hash_count
| N/A | N/A | 95.9 | 159 | 287 | 543 | 1,055 |
batch_insert_into_append_only_tree_32_depth_hash_ms
| N/A | N/A | 0.495 | 0.469 | 0.454 (
+1%
) | 0.444 | 0.440 |
batch_insert_into_indexed_tree_20_depth_ms
| N/A | N/A | 59.5 | 112 | 182 (
-1%
) | 353 | 698 |
batch_insert_into_indexed_tree_20_depth_hash_count
| N/A | N/A | 109 | 207 | 355 | 691 | 1,363 |
batch_insert_into_indexed_tree_20_depth_hash_ms
| N/A | N/A | 0.504 | 0.501 | 0.483 | 0.478 | 0.476 (
-1%
) |
batch_insert_into_indexed_tree_40_depth_ms
| N/A | N/A | 72.7 | N/A | N/A | N/A | N/A |
batch_insert_into_indexed_tree_40_depth_hash_count
| N/A | N/A | 133 | N/A | N/A | N/A | N/A |
batch_insert_into_indexed_tree_40_depth_hash_ms
| N/A | N/A | 0.518 | N/A | N/A | N/A | N/A | ### Miscellaneous Transaction sizes based on how many contract classes are registered in the tx. | Metric | 0 registered classes | 1 registered classes | | - | - | - |
tx_size_in_bytes
| 85,672 | 670,983 | Transaction size based on fee payment method | Metric | | | - | |
Benchmark results
Metrics with a significant change:
Detailed results
All benchmarks are run on txs on the `Benchmarking` contract on the repository. Each tx consists of a batch call to `create_note` and `increment_balance`, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write. This benchmark source data is available in JSON format on S3 [here](https://aztec-ci-artifacts.s3.us-east-2.amazonaws.com/benchmarks-v1/pulls/7268.json). ### Proof generation Each column represents the number of threads used in proof generation. | Metric | 1 threads | 4 threads | 16 threads | 32 threads | 64 threads | | - | - | - | - | - | - | proof_construction_time_sha256_30_ms | 11,712 (-2%) | 3,143 (-2%) | :warning: 1,437 (**-16%**) | 1,598 (+10%) | 1,591 (+2%) | proof_construction_time_sha256_100_ms | 44,251 | 11,760 (-1%) | 5,436 | 5,411 | 5,363 | proof_construction_time_poseidon_hash_ms | 77.0 (-3%) | 34.0 (-26%) | 34.0 (-3%) | 59.0 | 87.0 (-1%) | proof_construction_time_poseidon_hash_30_ms | 1,518 (-1%) | 416 (-1%) | 200 (+1%) | 226 (-1%) | 267 | proof_construction_time_poseidon_hash_100_ms | 5,762 (-1%) | 1,583 | 724 | 769 (-3%) | 798 (+1%) | ### L2 block published to L1 Each column represents the number of txs on an L2 block published to L1. | Metric | 4 txs | 8 txs | 16 txs | | - | - | - | - | l1_rollup_calldata_size_in_bytes | 1,412 | 1,412 | 1,412 | l1_rollup_calldata_gas | 9,464 | 9,464 | 9,464 | l1_rollup_execution_gas | 610,285 | 610,285 | 610,285 | l2_block_processing_time_in_ms | 762 | 1,409 (-1%) | 2,713 (-1%) | l2_block_building_time_in_ms | 25,231 (-1%) | 51,227 (+1%) | 99,188 (-1%) | l2_block_rollup_simulation_time_in_ms | 25,230 (-1%) | 51,226 (+1%) | 99,187 (-1%) | l2_block_public_tx_process_time_in_ms | 21,685 (-1%) | 47,453 (+1%) | 95,493 (-1%) | ### L2 chain processing Each column represents the number of blocks on the L2 chain where each block has 8 txs. | Metric | 3 blocks | 5 blocks | | - | - | - | node_history_sync_time_in_ms | 7,042 (-1%) | 9,846 (-1%) | node_database_size_in_bytes | 12,193,872 | 16,126,032 | pxe_database_size_in_bytes | 16,254 | 26,813 | ### Circuits stats Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks. | Circuit | simulation_time_in_ms | witness_generation_time_in_ms | proving_time_in_ms | input_size_in_bytes | output_size_in_bytes | proof_size_in_bytes | num_public_inputs | size_in_gates | | - | - | - | - | - | - | - | - | - | private-kernel-init | 129 (-2%) | 482 (+1%) | 13,161 (+1%) | 20,634 | 67,190 | 92,352 | 2,819 | 524,288 | private-kernel-inner | 392 | :warning: 1,145 (**+22%**) | 49,195 (-1%) | 94,902 | 67,190 | 92,352 | 2,819 | 2,097,152 | private-kernel-tail | 370 | 1,817 (-1%) | 51,716 (+2%) | 99,121 | 71,733 | 14,912 | 399 | 2,097,152 | base-parity | 6.15 | 1,870 (-4%) | 2,777 (+2%) | 128 | 64.0 | 2,208 | 2.00 | 131,072 | root-parity | 49.1 (+1%) | 77.2 (+5%) | 41,613 (-4%) | 27,100 | 64.0 | 2,720 | 18.0 | 2,097,152 | base-rollup | 7,933 | 4,920 | 83,812 (-7%) | 170,330 | 756 | 3,648 | 47.0 | 4,194,304 | root-rollup | 109 (-1%) | 91.2 (+3%) | 24,557 (+11%) | 25,309 | 620 | 3,456 | 41.0 | 1,048,576 | public-kernel-setup | 714 | 3,646 (-2%) | 45,633 (+1%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-app-logic | 612 | 4,693 (-2%) | 43,996 (-7%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-tail | 1,411 | 36,294 (-4%) | 194,219 (-1%) | 511,910 | 10,014 | 14,912 | 399 | 8,388,608 | private-kernel-reset-small | 593 | 2,171 (+3%) | 48,031 (+2%) | 123,313 | 67,190 | 92,352 | 2,819 | 2,097,152 | public-kernel-teardown | 612 | 4,678 | 46,623 (-1%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | merge-rollup | 28.6 | N/A | N/A | 16,542 | 756 | N/A | N/A | N/A | private-kernel-tail-to-public | N/A | 7,333 (-5%) | 99,409 (-1%) | N/A | N/A | 125,344 | 3,850 | 4,194,304 | Stats on running time collected for app circuits | Function | input_size_in_bytes | output_size_in_bytes | witness_generation_time_in_ms | proof_size_in_bytes | proving_time_in_ms | size_in_gates | num_public_inputs | | - | - | - | - | - | - | - | - | ContractClassRegisterer:register | 1,344 | 9,944 | 420 (-1%) | N/A | N/A | N/A | N/A | ContractInstanceDeployer:deploy | 1,408 | 9,944 | 39.4 (-1%) | N/A | N/A | N/A | N/A | MultiCallEntrypoint:entrypoint | 1,920 | 9,944 | 1,766 (-1%) | N/A | N/A | N/A | N/A | GasToken:deploy | 1,376 | 9,944 | 975 (-1%) | N/A | N/A | N/A | N/A | SchnorrAccount:constructor | 1,312 | 9,944 | 1,400 (-1%) | N/A | N/A | N/A | N/A | SchnorrAccount:entrypoint | 2,304 | 9,944 | 2,814 | 16,768 | 56,685 (+3%) | 2,097,152 | 457 | Token:privately_mint_private_note | 1,280 | 9,944 | 1,592 (-5%) | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_public | 1,344 | 9,944 | 353 | 16,768 | 11,193 | 524,288 | 457 | Token:transfer | 1,312 | 9,944 | 4,467 (-1%) | 16,768 | 45,704 (-5%) | 2,097,152 | 457 | AuthRegistry:set_authorized (avm) | 21,043 | N/A | N/A | 87,200 | 1,371 (+1%) | N/A | N/A | FPC:prepare_fee (avm) | 28,521 | N/A | N/A | 88,032 | 5,719 (-4%) | N/A | N/A | AuthRegistry:consume (avm) | 34,973 | N/A | N/A | 87,616 | 2,986 (-2%) | N/A | N/A | Token:transfer_public (avm) | 44,849 | N/A | N/A | 87,616 | 2,983 (-2%) | N/A | N/A | FPC:pay_refund (avm) | 33,381 | N/A | N/A | 88,448 | 11,006 (-4%) | N/A | N/A | Benchmarking:create_note | 1,344 | 9,944 | 1,395 (-1%) | N/A | N/A | N/A | N/A | SchnorrAccount:verify_private_authwit | 1,280 | 9,944 | 73.1 | N/A | N/A | N/A | N/A | Token:unshield | 1,376 | 9,944 | 3,688 | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_private | 1,376 | 9,944 | 4,621 (-1%) | N/A | N/A | N/A | N/A | ### AVM Simulation Time to simulate various public functions in the AVM. | Function | time_ms | bytecode_size_in_bytes | | - | - | - | GasToken:_increase_public_balance | 70.5 (-3%) | 13,873 | GasToken:set_portal | 17.0 (+1%) | 3,495 | Token:constructor | 90.2 (-2%) | 24,207 | FPC:constructor | 63.3 (-1%) | 13,893 | GasToken:mint_public | 52.7 (+3%) | 10,241 | Token:mint_public | :warning: 87.2 (**-86%**) | 19,216 | Token:assert_minter_and_mint | :warning: 217 (**+201%**) | 13,034 | AuthRegistry:set_authorized | 33.5 (+8%) | 7,869 | FPC:prepare_fee | 188 (+4%) | 15,187 | Token:transfer_public | :warning: 37.0 (**-39%**) | 31,425 | FPC:pay_refund | 137 (+10%) | 20,080 | Benchmarking:increment_balance | 2,696 | 15,465 | Token:_increase_public_balance | 56.2 (-1%) | 15,089 | FPC:pay_refund_with_shielded_rebate | 145 (-3%) | 21,167 | ### Public DB Access Time to access various public DBs. | Function | time_ms | | - | - | get-nullifier-index | 0.176 (+16%) | ### Tree insertion stats The duration to insert a fixed batch of leaves into each tree type. | Metric | 1 leaves | 16 leaves | 64 leaves | 128 leaves | 256 leaves | 512 leaves | 1024 leaves | | - | - | - | - | - | - | - | - | batch_insert_into_append_only_tree_16_depth_ms | 10.3 | 16.7 (-1%) | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_16_depth_hash_count | 16.8 | 31.7 | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_16_depth_hash_ms | 0.597 | 0.513 (-1%) | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_32_depth_ms | N/A | N/A | 48.5 | 76.4 | 132 (+1%) | 245 | 471 (-1%) | batch_insert_into_append_only_tree_32_depth_hash_count | N/A | N/A | 95.9 | 159 | 287 | 543 | 1,055 | batch_insert_into_append_only_tree_32_depth_hash_ms | N/A | N/A | 0.495 | 0.469 | 0.454 (+1%) | 0.444 | 0.440 | batch_insert_into_indexed_tree_20_depth_ms | N/A | N/A | 59.5 | 112 | 182 (-1%) | 353 | 698 | batch_insert_into_indexed_tree_20_depth_hash_count | N/A | N/A | 109 | 207 | 355 | 691 | 1,363 | batch_insert_into_indexed_tree_20_depth_hash_ms | N/A | N/A | 0.504 | 0.501 | 0.483 | 0.478 | 0.476 (-1%) | batch_insert_into_indexed_tree_40_depth_ms | N/A | N/A | 72.7 | N/A | N/A | N/A | N/A | batch_insert_into_indexed_tree_40_depth_hash_count | N/A | N/A | 133 | N/A | N/A | N/A | N/A | batch_insert_into_indexed_tree_40_depth_hash_ms | N/A | N/A | 0.518 | N/A | N/A | N/A | N/A | ### Miscellaneous Transaction sizes based on how many contract classes are registered in the tx. | Metric | 0 registered classes | 1 registered classes | | - | - | - | tx_size_in_bytes | 85,672 | 670,983 | Transaction size based on fee payment method | Metric | | | - | |