Basic benchmarking

[naturallymitchell]

Just to get a basic sense of this library's performance, I ran 4 tests using fibonacci (code included).

1. Lua 5.3.5

2. LuaDardo with same Lua code (AOT compiled)

3. Dart-native code (AOT compiled)

4. LuaDardo w/ bindings to native Dart code (AOT compiled)

~/luafib $ time lua test.lua 75025

real 0m0.062s user 0m0.048s sys 0m0.013s ~/luafib $ time lua test.lua 75025

real 0m0.061s user 0m0.044s sys 0m0.017s ~/luafib $ time lua test.lua 75025

real 0m0.065s user 0m0.048s sys 0m0.013s

2.

~/LuaDardo/example $ time ./example.exe 75025 real 0m1.225s user 0m1.194s sys 0m0.037s

~/LuaDardo/example $ time ./example.exe 75025 real 0m1.206s user 0m1.117s sys 0m0.094s

~/LuaDardo/example $ time ./example.exe 75025 real 0m1.252s user 0m1.190s sys 0m0.066s

3.

~/fib $ time ./fibonacci.exe 75025

real 0m0.071s user 0m0.018s sys 0m0.030s ~/fib $ time ./fibonacci.exe 75025

real 0m0.037s user 0m0.020s sys 0m0.017s ~/fib $ time ./fibonacci.exe 75025

real 0m0.040s user 0m0.015s sys 0m0.023s

4.

~/dfib $ time ./dfib.exe 75025

real 0m0.091s user 0m0.022s sys 0m0.038s ~/dfib $ time ./dfib.exe 75025

real 0m0.048s user 0m0.027s sys 0m0.017s ~/dfib $ time ./dfib.exe 75025

real 0m0.045s user 0m0.034s sys 0m0.010s

---

1.

function naive(n)
  local function inner(m) if m < 2 then return m end
    return inner(m-1) + inner(m-2) end return inner(n) end
print(naive(25))

2.

name: 'test'

environment:
  sdk: '>=2.16.2 <3.0.0'

dependencies:
  lua_dardo: ^0.0.4

import 'package:lua_dardo/lua.dart';

void main(List<String> arguments) {
  LuaState state = LuaState.newState();
  state.openLibs();
  state.loadString(r'''
function naive(n)
  local function inner(m)
    if m < 2 then
      return m
    end
    return inner(m-1) + inner(m-2)
  end
  return inner(n)
end
print(naive(25))
''');
  state.call(0, 0);
}

3.

import 'dart:io';

/// Find the nth term in the Fibonacci sequence
int fib(int n) {
  if (n < 2) {  //base case
    return n;
  }
  return fib(n - 2) + fib(n - 1);  //recursive case
}

void main() {
  print(fib(25));
}

4.

import 'package:lua_dardo/lua.dart';
import 'dart:io';

int fib25(LuaState ls) {
  int fib(int n) {
    if (n < 2) {
      return n;
    }
    return fib(n - 2) + fib(n - 1);
  }

// ls.pop(1);

  var fibVal = fib(25);
  ls.pushInteger(fibVal);
  return 1;
}

void main(List<String> arguments) {
  LuaState state = LuaState.newState();
  state.openLibs();

  state.pushDartFunction(fib25);
  state.setGlobal('fib25');

  state.loadString('''
fib_val = fib25()
print(fib_val)
''');
  state.call(0, 0);
}

[maks]

@naturallymitchell Thanks for doing these tests.

Micro benchmarks are notorious, but it doesn't sound quite right that the Lua 5.3.5 interpreter (I'm assuming you weren't using LuaJit?) is within even 20-30% the speed for aot compiled Dart code. Trying your test code with Dart AOT and Lua 5.3.6 on Ubuntu 22.04, I got comparatively same results as you posted. So I suspected that maybe the startup time is swamping the actual results.

Trying your test code with higher iteration counts I get very different comparative results for Dart AOT vs Lua 5.3.1:

with 30:

time lib/main.exe 
832040

real    0m0.013s
user    0m0.009s
sys     0m0.005s

time lua bench.lua 
832040

real    0m0.068s
user    0m0.068s
sys     0m0.000s

with 40:

time lib/main.exe 
102334155

real    0m0.588s
user    0m0.580s
sys     0m0.008s

time lua bench.lua 
102334155

real    0m8.050s
user    0m8.046s
sys     0m0.004s

with 50:

time lib/main.exe 
12586269025

real    1m12.875s
user    1m12.870s
sys     0m0.004s
time lua bench.lua 
12586269025

real    16m15.980s
user    16m15.896s
sys     0m0.016s

I'll try to make some time to run the microbenchmark for the higher iterations with LuaDardo and the native Dart binding soon too.