use mops for all motoko benchmarks

[chenyan-dfinity]

• This also allows to tune the build script for Motoko and Rust separately. Helps the skewness of #83
• Bump sha2 to 0.0.4

[github-actions[bot]]

Note Diffing the performance result against the published result from main branch. Unchanged benchmarks are omitted.

SHA-2

	binary_size	SHA-256	SHA-512	account_id	neuron_id
Motoko	172_890 ($\textcolor{green}{-0.98\%}$)	247_480_401 ($\textcolor{green}{-6.31\%}$)	228_033_044 ($\textcolor{green}{-3.01\%}$)	30_017 ($\textcolor{green}{-14.61\%}$)	20_760 ($\textcolor{green}{-10.76\%}$)
Rust	498_272	82_512_024	56_525_977	42_551	44_574

Certified map

	binary_size	generate 10k	max mem	inc	witness
Motoko	176_829 ($\textcolor{green}{-0.81\%}$)	4_390_018_085 ($\textcolor{green}{-12.44\%}$)	3_429_924	519_711 ($\textcolor{green}{-12.53\%}$)	327_767
Rust	441_794	6_202_163_062	1_081_344	983_904	288_469

Statistics

• binary_size: -0.89% [-1.42%, -0.36%]
• max_mem: no change
• cycles: -9.94% [-13.56%, -6.33%]

Basic DAO

	binary_size	init	transfer_token	submit_proposal	vote_proposal
Motoko	230_182 ($\textcolor{green}{-0.03\%}$)	37_614 ($\textcolor{red}{0.06\%}$)	16_290 ($\textcolor{green}{-0.31\%}$)	12_672	14_136 ($\textcolor{red}{0.16\%}$)
Rust	718_379	472_385	86_786	105_263	116_229

DIP721 NFT

Note Same as main branch, skipping.

Statistics

• binary_size: -0.03%
• max_mem: no change
• cycles: -0.03% [-0.44%, 0.38%]

Heartbeat

	binary_size	heartbeat
Motoko	123_357	3_758
Rust	23_625	785 ($\textcolor{red}{67.38\%}$)

Timer

Note Same as main branch, skipping.

Statistics

• binary_size: no change

• max_mem: no change

• cycles: no change

  Overall Statistics

• binary_size: -0.60% [-1.46%, 0.25%]

• max_mem: no change

• cycles: -6.64% [-10.39%, -2.88%]

[github-actions[bot]]

Note The flamegraph link only works after you merge. Unchanged benchmarks are omitted.

Cryptographic libraries

Measure different cryptographic libraries written in both Motoko and Rust.

• SHA-2 benchmarks
  • SHA-256/SHA-512. Compute the hash of a 1M Wasm binary.
  • account_id. Compute the ledger account id from principal, based on SHA-224.
  • neuron_id. Compute the NNS neuron id from principal, based on SHA-256.
• Certified map. Merkle Tree for storing key-value pairs and generate witness according to the IC Interface Specification.
  • generate 10k. Insert 10k 7-character word as both key and value into the certified map.
  • max mem. For Motoko, it reports rts_max_heap_size after generate call; For Rust, it reports the Wasm's memory page * 32Kb.
  • inc. Increment a counter and insert the counter value into the map.
  • witness. Generate the root hash and a witness for the counter.

SHA-2

	binary_size	SHA-256	SHA-512	account_id	neuron_id
Motoko	172_890	247_480_401	228_033_044	30_017	20_760
Rust	498_272	82_512_024	56_525_977	42_551	44_574

Certified map

	binary_size	generate 10k	max mem	inc	witness
Motoko	176_829	4_390_018_085	3_429_924	519_711	327_767
Rust	441_794	6_202_163_062	1_081_344	983_904	288_469

Sample Dapps

Measure the performance of some typical dapps:

• Basic DAO, with heartbeat disabled to make profiling easier. We have a separate benchmark to measure heartbeat performance.
• DIP721 NFT

Note

• The cost difference is mainly due to the Candid serialization cost.
• Motoko statically compiles/specializes the serialization code for each method, whereas in Rust, we use serde to dynamically deserialize data based on data on the wire.
• We could improve the performance on the Rust side by using parser combinators. But it is a challenge to maintain the ergonomics provided by serde.
• For real-world applications, we tend to send small data for each endpoint, which makes the Candid overhead in Rust tolerable.

Basic DAO

	binary_size	init	transfer_token	submit_proposal	vote_proposal
Motoko	230_182	37_614	16_290	12_672	14_136
Rust	718_379	472_385	86_786	105_263	116_229

DIP721 NFT

	binary_size	init	mint_token	transfer_token
Motoko	188_321	12_267	22_357	4_729
Rust	778_280	125_293	325_017	77_500

Heartbeat / Timer

Measure the cost of empty heartbeat and timer job.

• setTimer measures both the setTimer(0) method and the execution of empty job.
• It is not easy to reliably capture the above events in one flamegraph, as the implementation detail of the replica can affect how we measure this. Typically, a correct flamegraph contains both setTimer and canister_global_timer function. If it's not there, we may need to adjust the script.

Heartbeat

	binary_size	heartbeat
Motoko	123_357	3_758
Rust	23_625	785

Timer

	binary_size	setTimer	cancelTimer
Motoko	129_636	15_227	1_684
Rust	443_367	43_417	7_497