search

lcompilers / lc

C++ compiler
MIT License
26 stars 7 forks source link

Some benchmarks #38

Open certik opened 10 months ago

certik commented 10 months ago

Here are some benchmark results.

array_09.cpp

Source code: https://github.com/lcompilers/lc/blob/f6548855009827d57398f5335ca629b712b1a8be/integration_tests/array_09.cpp

An equivalent Fortran code: https://github.com/lfortran/lfortran/blob/0425cf2c0b988762db49033bfa9696483c345c69/integration_tests/arrays_op_17.f90

Using Apple M1 Max.

Compilation Speed

$ time clang++ -I$CONDA_PREFIX/include integration_tests/array_09.cpp -o x.clang
clang++ -I$CONDA_PREFIX/include integration_tests/array_09.cpp -o x.clang  0.99s user 0.07s system 100% cpu 1.048 total
$ time lc --extra-arg-before="-I$CONDA_PREFIX/include" integration_tests/array_09.cpp -o x.lc
Error while trying to load a compilation database:
Could not auto-detect compilation database for file "integration_tests/array_09.cpp"
No compilation database found in /Users/ondrej/repos/lcompilers/lc/integration_tests or any parent directory
fixed-compilation-database: Error while opening fixed database: No such file or directory
json-compilation-database: Error while opening JSON database: No such file or directory
Running without flags.
lc --extra-arg-before="-I$CONDA_PREFIX/include" integration_tests/array_09.cp  0.71s user 0.05s system 98% cpu 0.772 total

LC is 1.36x faster.

For a comparison, LFortran compiles the equivalent code in:

$ time lfortran arrays_op_17.f90 -o x.lf
lfortran arrays_op_17.f90 -o x.lf  0.06s user 0.01s system 95% cpu 0.076 total

So another 10x speedup is possible by using our own custom parser for a subset of C++.

Further speedup is possible by not using LLVM.

Binary Size

$ ll -h x.clang 
-rwxr-xr-x  1 ondrej  staff   1.2M Dec 21 16:32 x.clang
$ strip x.clang 
$ ll -h x.clang
-rwxr-xr-x  1 ondrej  staff   1.0M Dec 21 16:32 x.clang
$ ll -h x.lc
-rwxr-xr-x  1 ondrej  staff    33K Dec 21 16:29 x.lc

LC generates 32.3x smaller binary.

Runtime Speed

$ time ./x.clang
{2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2}{{4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4},
 {4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4}}{8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8}
{{34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34},
 {34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34,
  34, 34}}
./x.clang  0.01s user 0.00s system 78% cpu 0.014 total
$ time ./x.lc
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 

8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 
34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 34 

./x.lc  0.00s user 0.00s system 62% cpu 0.008 total

LC binary runs 1.75x faster.