search

demergent-labs / canister-profiling

Collection of canister performance benchmarks
Apache License 2.0
0 stars 0 forks source link

Heartbeat Benchmarks #1

Open dansteren opened 1 year ago

dansteren commented 1 year ago

Implement the heartbeat example from https://github.com/demergent-labs/canister-profiling/tree/main/heartbeat in Azle v0.18.2 syntax.

dansteren commented 1 year ago

I'm not seeing strong differences between the new and old instrumentation. Here's the forum post where the new instrumentation is announced: https://forum.dfinity.org/t/new-wasm-instrumentation/22080

Here's what I'm getting so far:

Heartbeat

binary_size heartbeat (dfx v0.14.4) heartbeat (dfx v0.15.0)
Motoko 134_224 5_374 4_120
Rust 25_642 468 789
Azle@0.17.1 5_524_252 95_688 95_688
Azle@0.18.0 5_876_339 13_264 13_913

Timer

binary_size setTimer (dfx v0.14.4) setTimer (dfx v0.15.0) cancelTimer (dfx v0.14.4) cancelTimer (dfx v0.15.0)
Motoko 141_727 19_749 16_462 1_920 1_844
Rust 472_532 44_880 44_880 7_736 7_736
Azle@0.17.1 5_583_479 493_586 493_586 164_530 164_530
Azle@0.18.0 5_876_668 3_291_322 3_291_367 2_362_633 2_361_699
dansteren commented 1 year ago

Kybra is crazy because we make an uploader canister, and then call a post_install script to chunk upload the python stdlib and the actual canister. Getting it to work in this framework is going to be very challenging. I'm not even sure what arbitrary commands you can run from within an ic-repl script. The perf.sh script is what's currently doing the installation so it feels like we need it in there. There's a chance we could do it in the makefile before we get to perf.sh, or go back and forth in between the two somehow.

Either way, getting kybra to be measured here will not be trivial.

dansteren commented 1 year ago

Note that before running the benchmarks you must: