on Lispkits performance

Hi, I just run a simple and very limited benchmark to get idea where Lispkit score in terms of and startup and runtime performance. Below there are some mesurements where the test shows that Lispkit is slower.

Can you say something on Lispkits performance? Are there specific reasons? Is my testing setup wrong?

Thanks

Simple hello.scm

(display "hellooo")

Lispkit

time .build/debug/LispKitRepl -r Sources/LispKit/Resources -d LispKit hello.scm
hellooo
________________________________________________________
Executed in  689.86 millis    fish           external
   usr time  651.39 millis    0.20 millis  651.18 millis
   sys time   26.63 millis    1.53 millis   25.09 millis

Gauche

time gosh hello.scm
hellooo
________________________________________________________
Executed in   59.41 millis    fish           external
   usr time   28.57 millis    0.17 millis   28.40 millis
   sys time   18.11 millis    1.26 millis   16.85 millis

Simple fib.scm

(define (fib n)
  (if (<= n 1) n
      (+ (fib (- n 1))
           (fib (- n 2)))))

(display (fib 30))
(newline)

Lispkit

time .build/debug/LispKitRepl -r Sources/LispKit/Resources -d LispKit fib.scm
75025

________________________________________________________
Executed in    4.62 secs    fish           external
   usr time    4.53 secs    0.18 millis    4.53 secs
   sys time    0.05 secs    1.49 millis    0.05 secs

Gosh

time gosh fib.scm
75025

________________________________________________________
Executed in   32.94 millis    fish           external
   usr time   32.40 millis    0.17 millis   32.23 millis
   sys time   15.04 millis    1.47 millis   13.57 millis

If you care about performance then LispKit isn't really the right tool as it inherits quite a few performance penalties from Swift itself (e.g. it uses enumerations with associated values extensively which tend to be quite slow and also ARC as garbage collector). But if you do want to benchmark it, you should definitely only use release binaries. The difference is significant.

You can create a release binary with make release. This is what I get with release binaries for hello.scm:

[LispKit/swift-lispkit] time .build/release/LispKitRepl -r Sources/LispKit/Resources --- hello.scm
hellooo
================
CPU 91%
user    67ms
system  14ms
total   88ms

[LispKit/swift-lispkit] time gosh hello.scm
hellooo
================
CPU 142%
user    43ms
system  34ms
total   54ms

And these are the results for fib.scm:

[LispKit/swift-lispkit] time .build/release/LispKitRepl -r Sources/LispKit/Resources --- fib.scm
75025
================
CPU 96%
user    332ms
system  18ms
total   361ms

[LispKit/swift-lispkit] time gosh fib.scm 
75025
================
CPU 161%
user    51ms
system  22ms
total   45ms

The results are exactly what I would expect: LispKit should have a similar startup time compared to Gauche. But anything computationally expensive will be way slower compared to an optimizing compiler like Gauche (at least an order of magnitude). You can see that a significant amount of time is spent in the mathematical functions (which are all automatically turned into native integer arithmetic by Gauche) by replacing LispKit's polymorphic arithmetics with plain fixnum operations:

(define (fib n)
  (if (fx<= n 1) n
      (fx+ (fib (fx1- n))
             (fib (fx- n 2)))))
(display (fib 25))
(newline)

Here's how that performs:

[LispKit/swift-lispkit] time .build/release/LispKitRepl -r Sources/LispKit/Resources --- fib2.scm
75025
================
CPU 96%
user    200ms
system  14ms
total   223ms

Another interesting observation — and this is why I gave up running microbenchmarks long ago — is that the performance of code compiled with Xcode IDE is significantly better than with the command-line tools. Using the time procedure from library (lispkit debug), I get the following via LispKitRepl:

> (time (fib 30))
elapsed time = 1.4443590420014516
832040

And this is what I get with the latest version of LispPad (compiled with the Xcode IDE):

➤ (time (fib 30))
elapsed time = 1.0428437919981661
832040

objecthub / swift-lispkit