feat: TXE 2: Electric boogaloo

[Thunkar]

Adds new capabilities to TXE, and a proper TokenContract suite to showcase them:

• Private -> Public calls
• Public -> Public calls
• Actual account contract deployment for complex interactions
• Auhtwits, both public and private
• Note cache storage
• Speed improvements

The full test suite is comparable to the e2e one (with some caveats, some things just cannot be tested yet using TXE), but runs significantly faster. Interestingly, runs faster on an M3 macbook than on mainframe (120s vs. 240s).

e2e tests take more than 600s (without stored snapshots), but the gap increases as we reduce the amount of tests launched simultaneously. An unconstrained private transfer runs in less than 10s using TXE!

If launching many tests in parallel, it is recommended to increase the nargo timeout for foreign calls to avoid timeout errors: NARGO_FOREIGN_CALL_TIMEOUT=300000

Closes https://github.com/AztecProtocol/aztec-packages/issues/7086

[AztecBot]

Benchmark results

Metrics with a significant change:

• avm_simulation_time_ms (Token:mint_public): 70.6 (-88%)
• avm_simulation_time_ms (Token:assert_minter_and_mint): 211 (+187%)
• protocol_circuit_witness_generation_time_in_ms (private-kernel-inner): 1,123 (+17%)

Detailed results

All benchmarks are run on txs on the `Benchmarking` contract on the repository. Each tx consists of a batch call to `create_note` and `increment_balance`, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write. This benchmark source data is available in JSON format on S3 [here](https://aztec-ci-artifacts.s3.us-east-2.amazonaws.com/benchmarks-v1/pulls/7154.json). ### Proof generation Each column represents the number of threads used in proof generation. | Metric | 1 threads | 4 threads | 16 threads | 32 threads | 64 threads | | - | - | - | - | - | - | proof_construction_time_sha256_30_ms | 11,689 | 3,137 (-2%) | 1,695 (-4%) | 1,461 (-11%) | 1,572 (+8%) | proof_construction_time_sha256_100_ms | 44,758 | 11,743 | 5,439 | 5,428 | 5,367 | proof_construction_time_poseidon_hash_ms | 78.0 | 34.0 | 34.0 | 57.0 (-2%) | 89.0 | proof_construction_time_poseidon_hash_30_ms | 1,513 | 415 | 199 | 221 (-3%) | 270 (+1%) | proof_construction_time_poseidon_hash_100_ms | 5,725 | 1,574 (-1%) | 735 (+1%) | 770 (+1%) | 792 | ### L2 block published to L1 Each column represents the number of txs on an L2 block published to L1. | Metric | 4 txs | 8 txs | 16 txs | | - | - | - | - | l1_rollup_calldata_size_in_bytes | 1,412 | 1,412 | 1,412 | l1_rollup_calldata_gas | 9,476 | 9,472 | 9,476 | l1_rollup_execution_gas | 610,297 | 610,293 | 610,297 | l2_block_processing_time_in_ms | 751 | 1,400 | 2,673 (-1%) | l2_block_building_time_in_ms | 25,398 (-1%) | 51,420 | 99,087 (-2%) | l2_block_rollup_simulation_time_in_ms | 25,302 (-2%) | 51,111 | 98,714 (-2%) | l2_block_public_tx_process_time_in_ms | 21,734 (-2%) | 47,314 | 95,021 (-2%) | ### L2 chain processing Each column represents the number of blocks on the L2 chain where each block has 8 txs. | Metric | 3 blocks | 5 blocks | | - | - | - | node_history_sync_time_in_ms | 6,900 (-2%) | 9,818 | node_database_size_in_bytes | 12,202,064 | 16,085,072 | pxe_database_size_in_bytes | 16,254 | 26,813 | ### Circuits stats Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks. | Circuit | simulation_time_in_ms | witness_generation_time_in_ms | proving_time_in_ms | input_size_in_bytes | output_size_in_bytes | proof_size_in_bytes | num_public_inputs | size_in_gates | | - | - | - | - | - | - | - | - | - | private-kernel-init | 131 (+2%) | 470 | 13,202 (-3%) | 20,634 | 67,190 | 92,352 | 2,819 | 524,288 | private-kernel-inner | 396 (+1%) | :warning: 1,123 (**+17%**) | 50,174 (-2%) | 94,902 | 67,190 | 92,352 | 2,819 | 2,097,152 | private-kernel-tail | 372 | 1,824 (-2%) | 51,546 (-4%) | 99,121 | 71,733 | 14,912 | 399 | 2,097,152 | base-parity | 6.25 | 1,894 (-3%) | 2,729 | 128 | 64.0 | 2,208 | 2.00 | 131,072 | root-parity | 49.1 | 72.9 (-5%) | 39,909 (-2%) | 27,100 | 64.0 | 2,720 | 18.0 | 2,097,152 | base-rollup | 7,915 (-1%) | 4,974 (+1%) | 85,491 (-2%) | 170,330 | 756 | 3,648 | 47.0 | 4,194,304 | root-rollup | 111 (+2%) | 85.1 (-5%) | 23,411 (+4%) | 25,309 | 620 | 3,456 | 41.0 | 1,048,576 | public-kernel-setup | 718 (+1%) | 3,756 | 45,316 | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-app-logic | 611 | 4,777 | 47,095 (-1%) | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | public-kernel-tail | 1,417 | 37,856 (+8%) | 192,473 | 511,910 | 10,014 | 14,912 | 399 | 8,388,608 | private-kernel-reset-small | 593 | 2,009 (-1%) | 46,612 (-1%) | 123,313 | 67,190 | 92,352 | 2,819 | 2,097,152 | public-kernel-teardown | 616 (+1%) | 4,784 (+2%) | 47,266 | 116,905 | 93,334 | 125,344 | 3,850 | 2,097,152 | merge-rollup | 29.2 (+1%) | N/A | N/A | 16,542 | 756 | N/A | N/A | N/A | private-kernel-tail-to-public | N/A | 6,893 (-1%) | 100,088 (-2%) | N/A | N/A | 125,344 | 3,850 | 4,194,304 | Stats on running time collected for app circuits | Function | input_size_in_bytes | output_size_in_bytes | witness_generation_time_in_ms | proof_size_in_bytes | proving_time_in_ms | size_in_gates | num_public_inputs | | - | - | - | - | - | - | - | - | ContractClassRegisterer:register | 1,344 | 9,944 | 424 | N/A | N/A | N/A | N/A | ContractInstanceDeployer:deploy | 1,408 | 9,944 | 40.1 (+2%) | N/A | N/A | N/A | N/A | MultiCallEntrypoint:entrypoint | 1,920 | 9,944 | 1,786 | N/A | N/A | N/A | N/A | GasToken:deploy | 1,376 | 9,944 | 988 (+2%) | N/A | N/A | N/A | N/A | SchnorrAccount:constructor | 1,312 | 9,944 | 1,415 | N/A | N/A | N/A | N/A | SchnorrAccount:entrypoint | 2,304 | 9,944 | 2,848 (+1%) | 16,768 | 57,530 (+4%) | 2,097,152 | 457 | Token:privately_mint_private_note | 1,280 | 9,944 | 1,639 | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_public | 1,344 | 9,944 | 358 (+4%) | 16,768 | 10,985 (-2%) | 524,288 | 457 | Token:transfer | 1,312 | 9,944 | 4,522 (+3%) | 16,768 | 46,765 | 2,097,152 | 457 | AuthRegistry:set_authorized (avm) | 21,043 | N/A | N/A | 87,200 | 1,385 (+2%) | N/A | N/A | FPC:prepare_fee (avm) | 28,841 | N/A | N/A | 88,032 | 5,871 | N/A | N/A | Token:transfer_public (avm) | 44,971 | N/A | N/A | 87,865 | 4,782 (-1%) | N/A | N/A | AuthRegistry:consume (avm) | 34,973 | N/A | N/A | 87,616 | 3,202 (+7%) | N/A | N/A | FPC:pay_refund (avm) | 33,573 | N/A | N/A | 88,448 | 11,405 | N/A | N/A | Benchmarking:create_note | 1,344 | 9,944 | 1,422 (+1%) | N/A | N/A | N/A | N/A | SchnorrAccount:verify_private_authwit | 1,280 | 9,944 | 77.7 (+7%) | N/A | N/A | N/A | N/A | Token:unshield | 1,376 | 9,944 | 3,773 (+6%) | N/A | N/A | N/A | N/A | FPC:fee_entrypoint_private | 1,376 | 9,944 | 4,737 (+2%) | N/A | N/A | N/A | N/A | ### AVM Simulation Time to simulate various public functions in the AVM. | Function | time_ms | bytecode_size_in_bytes | | - | - | - | GasToken:_increase_public_balance | 66.5 (+2%) | 13,873 | GasToken:set_portal | 17.2 (-6%) | 3,495 | Token:constructor | 91.0 (-1%) | 24,207 | FPC:constructor | 63.4 | 13,893 | GasToken:mint_public | 52.2 (+2%) | 10,241 | Token:mint_public | :warning: 70.6 (**-88%**) | 19,216 | Token:assert_minter_and_mint | :warning: 211 (**+187%**) | 13,034 | AuthRegistry:set_authorized | 32.5 (+7%) | 7,869 | FPC:prepare_fee | 149 (-2%) | 15,187 | Token:transfer_public | 50.1 (-4%) | 31,425 | FPC:pay_refund | 147 (+1%) | 20,080 | Benchmarking:increment_balance | 2,680 (-1%) | 15,465 | Token:_increase_public_balance | 56.1 | 15,089 | FPC:pay_refund_with_shielded_rebate | 146 (-1%) | 21,167 | ### Public DB Access Time to access various public DBs. | Function | time_ms | | - | - | get-nullifier-index | 0.157 (+2%) | ### Tree insertion stats The duration to insert a fixed batch of leaves into each tree type. | Metric | 1 leaves | 16 leaves | 64 leaves | 128 leaves | 256 leaves | 512 leaves | 1024 leaves | | - | - | - | - | - | - | - | - | batch_insert_into_append_only_tree_16_depth_ms | 10.3 | 16.7 (+1%) | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_16_depth_hash_count | 16.8 | 31.7 | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_16_depth_hash_ms | 0.596 | 0.513 | N/A | N/A | N/A | N/A | N/A | batch_insert_into_append_only_tree_32_depth_ms | N/A | N/A | 48.9 (+1%) | 75.9 | 131 (-1%) | 244 | 466 (-1%) | batch_insert_into_append_only_tree_32_depth_hash_count | N/A | N/A | 95.9 | 159 | 287 | 543 | 1,055 | batch_insert_into_append_only_tree_32_depth_hash_ms | N/A | N/A | 0.499 (+2%) | 0.466 | 0.450 | 0.442 | 0.435 (-1%) | batch_insert_into_indexed_tree_20_depth_ms | N/A | N/A | 59.9 (+1%) | 111 (-1%) | 180 (-1%) | 350 (-1%) | 689 | batch_insert_into_indexed_tree_20_depth_hash_count | N/A | N/A | 109 | 207 | 355 | 691 | 1,363 | batch_insert_into_indexed_tree_20_depth_hash_ms | N/A | N/A | 0.506 (+1%) | 0.499 | 0.476 (-1%) | 0.475 | 0.473 | batch_insert_into_indexed_tree_40_depth_ms | N/A | N/A | 72.9 | N/A | N/A | N/A | N/A | batch_insert_into_indexed_tree_40_depth_hash_count | N/A | N/A | 133 | N/A | N/A | N/A | N/A | batch_insert_into_indexed_tree_40_depth_hash_ms | N/A | N/A | 0.519 | N/A | N/A | N/A | N/A | ### Miscellaneous Transaction sizes based on how many contract classes are registered in the tx. | Metric | 0 registered classes | 1 registered classes | | - | - | - | tx_size_in_bytes | 85,672 | 670,983 | Transaction size based on fee payment method | Metric | | | - | |