refactor makefile

[github-actions[bot]]

Note Diffing the performance result against the published result from main branch. Unchanged benchmarks are omitted.

Basic DAO

	binary_size	init	transfer_token	submit_proposal	vote_proposal
Motoko	230_242	37_638 ($\textcolor{red}{0.06\%}$)	16_286 ($\textcolor{green}{-0.02\%}$)	12_714	14_163 ($\textcolor{green}{-0.16\%}$)
Rust	718_379	472_385	86_786 ($\textcolor{green}{-0.06\%}$)	105_263	116_229

DIP721 NFT

Note Same as main branch, skipping.

Statistics

• binary_size: no change
• max_mem: no change
• cycles: -0.04% [-0.15%, 0.06%]

Heartbeat

	binary_size	heartbeat
Motoko	123_357	3_758 ($\textcolor{green}{-49.21\%}$)
Rust	23_625	785

Timer

Note Same as main branch, skipping.

Statistics

• binary_size: no change

• max_mem: no change

• cycles: no change

  Overall Statistics

• binary_size: no change

• max_mem: no change

• cycles: -0.04% [-0.15%, 0.06%]

[github-actions[bot]]

Note The flamegraph link only works after you merge. Unchanged benchmarks are omitted.

Sample Dapps

Measure the performance of some typical dapps:

• Basic DAO, with heartbeat disabled to make profiling easier. We have a separate benchmark to measure heartbeat performance.
• DIP721 NFT

Note

• The cost difference is mainly due to the Candid serialization cost.
• Motoko statically compiles/specializes the serialization code for each method, whereas in Rust, we use serde to dynamically deserialize data based on data on the wire.
• We could improve the performance on the Rust side by using parser combinators. But it is a challenge to maintain the ergonomics provided by serde.
• For real-world applications, we tend to send small data for each endpoint, which makes the Candid overhead in Rust tolerable.

Basic DAO

	binary_size	init	transfer_token	submit_proposal	vote_proposal
Motoko	230_242	37_638	16_286	12_714	14_163
Rust	718_379	472_385	86_786	105_263	116_229

DIP721 NFT

	binary_size	init	mint_token	transfer_token
Motoko	188_321	12_267	22_357	4_729
Rust	778_280	125_293	325_017	77_500

Heartbeat / Timer

Measure the cost of empty heartbeat and timer job.

• setTimer measures both the setTimer(0) method and the execution of empty job.
• It is not easy to reliably capture the above events in one flamegraph, as the implementation detail of the replica can affect how we measure this. Typically, a correct flamegraph contains both setTimer and canister_global_timer function. If it's not there, we may need to adjust the script.

Heartbeat

	binary_size	heartbeat
Motoko	123_357	3_758
Rust	23_625	785

Timer

	binary_size	setTimer	cancelTimer
Motoko	129_636	15_227	1_684
Rust	443_367	43_417	7_497

[chenyan-dfinity]

That's also new to me! ChatGPT told me so :)