The Cyclone compiler memoizes the results of certain functions by default.
This is a very specialized optimization that actually changes the algorithm used
(turning fib into a loop, for example) so the benchmark doesn't measure for
Cyclone what it measures for all other Schemes, i.e., the speed of simple function calls.
This leads to the following results on my machine:
Testing fib under Cyclone
Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm
Compiling...
cyclone_comp /tmp/larcenous/Cyclone/fib.scm /tmp/larcenous/Cyclone/fib.scm
Running...
Running fib:40:5
Elapsed time: 0.000248 seconds (0.0) for fib:40:5
+!CSVLINE!+cyclone-0.26,fib:40:5,0.000248
real 0m0.003s
user 0m0.001s
sys 0m0.002s
Testing fibfp under Cyclone
Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm
Compiling...
cyclone_comp /tmp/larcenous/Cyclone/fibfp.scm /tmp/larcenous/Cyclone/fibfp.scm
Running...
Running fibfp:35.0:10
Elapsed time: 0.000243 seconds (0.0) for fibfp:35.0:10
+!CSVLINE!+cyclone-0.26,fibfp:35.0:10,0.000243
real 0m0.003s
user 0m0.003s
sys 0m0.000s
Testing ack under Cyclone
Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm
Compiling...
cyclone_comp /tmp/larcenous/Cyclone/ack.scm /tmp/larcenous/Cyclone/ack.scm
Running...
Running ack:3:12:2
Elapsed time: 0.196042 seconds (0.0) for ack:3:12:2
+!CSVLINE!+cyclone-0.26,ack:3:12:2,0.196042
real 0m0.201s
user 0m0.186s
sys 0m0.020s
An experimental fluid dynamicist I respected told me as a student that there are two parts
to each experiment: (1) Figure out precisely what it is that you want to measure, and
(2) figure out precisely how to measure it.
I think his advice applies in this situation: To compare speeds of function calls across
implementations, I think it would be best to add -no-memoization-optimizations
to the compilation flags of all Cyclone runs.
On my machine adding -no-memoization-optimizations to Cyclone's compilation flags yields:
ecraven
commented
1 month ago
The Cyclone compiler memoizes the results of certain functions by default. This is a very specialized optimization that actually changes the algorithm used (turning fib into a loop, for example) so the benchmark doesn't measure for Cyclone what it measures for all other Schemes, i.e., the speed of simple function calls.
This leads to the following results on my machine:
heine:~/programs/r7rs-benchmarks> ./bench cyclone "fib fibfp ack"
Testing fib under Cyclone Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm Compiling... cyclone_comp /tmp/larcenous/Cyclone/fib.scm /tmp/larcenous/Cyclone/fib.scm Running... Running fib:40:5 Elapsed time: 0.000248 seconds (0.0) for fib:40:5 +!CSVLINE!+cyclone-0.26,fib:40:5,0.000248
real 0m0.003s user 0m0.001s sys 0m0.002s
Testing fibfp under Cyclone Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm Compiling... cyclone_comp /tmp/larcenous/Cyclone/fibfp.scm /tmp/larcenous/Cyclone/fibfp.scm Running... Running fibfp:35.0:10 Elapsed time: 0.000243 seconds (0.0) for fibfp:35.0:10 +!CSVLINE!+cyclone-0.26,fibfp:35.0:10,0.000243
real 0m0.003s user 0m0.003s sys 0m0.000s
Testing ack under Cyclone Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm Compiling... cyclone_comp /tmp/larcenous/Cyclone/ack.scm /tmp/larcenous/Cyclone/ack.scm Running... Running ack:3:12:2 Elapsed time: 0.196042 seconds (0.0) for ack:3:12:2 +!CSVLINE!+cyclone-0.26,ack:3:12:2,0.196042
real 0m0.201s user 0m0.186s sys 0m0.020s
An experimental fluid dynamicist I respected told me as a student that there are two parts to each experiment: (1) Figure out precisely what it is that you want to measure, and (2) figure out precisely how to measure it.
I think his advice applies in this situation: To compare speeds of function calls across implementations, I think it would be best to add -no-memoization-optimizations to the compilation flags of all Cyclone runs.
On my machine adding -no-memoization-optimizations to Cyclone's compilation flags yields:
heine:~/programs/r7rs-benchmarks> ./bench cyclone "fib fibfp ack"
Testing fib under Cyclone Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm Compiling... cyclone_comp /tmp/larcenous/Cyclone/fib.scm /tmp/larcenous/Cyclone/fib.scm Running... Running fib:40:5 Elapsed time: 23.192617 seconds (24.0) for fib:40:5 +!CSVLINE!+cyclone-0.26,fib:40:5,23.192617
real 0m23.196s user 0m23.193s sys 0m0.008s
Testing fibfp under Cyclone Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm Compiling... cyclone_comp /tmp/larcenous/Cyclone/fibfp.scm /tmp/larcenous/Cyclone/fibfp.scm Running... Running fibfp:35.0:10 Elapsed time: 5.764464 seconds (6.0) for fibfp:35.0:10 +!CSVLINE!+cyclone-0.26,fibfp:35.0:10,5.764464
real 0m5.768s user 0m5.761s sys 0m0.008s
Testing ack under Cyclone Including postlude /home/lucier/programs/r7rs-benchmarks/src/Cyclone-postlude.scm Compiling... cyclone_comp /tmp/larcenous/Cyclone/ack.scm /tmp/larcenous/Cyclone/ack.scm Running... Running ack:3:12:2 Elapsed time: 34.968563 seconds (35.0) for ack:3:12:2 +!CSVLINE!+cyclone-0.26,ack:3:12:2,34.968563
real 0m35.002s user 0m34.699s sys 0m0.288s