search

nordlow / compiler-benchmark

Benchmarks compilation speeds of different combinations of languages and compilers.
MIT License
138 stars 18 forks source link

Compiler options are not comparable #17

Closed ufechner7 closed 3 years ago

ufechner7 commented 3 years ago

If you want to compare the compilation times of gcc and Julia the compiler optimization options should be comparable.

Julia is running by default with -O2, gcc by default with -O0

The benchmarks should either use -O2 for gcc or the option --compile=min for Julia.

nordlow commented 3 years ago

Ahh, thanks! I'll update. Hang on!

nordlow commented 3 years ago

For

./benchmark --languages=Julia --function-count=50 --function-depth=50

this gives a 5x speedup. Thanks.

PallHaraldsson commented 3 years ago

Since you merged this, and are ok with it, i.e. the purpose of the benchmark to time fastest (debug) build(?) then, then can you change the readme too? E.g. "100280.5x" looks bad.

I would like -O0 in Julia to actually mean no optimization. The other option means the interpreter is used (I still think "compiled", i.e. then LLVM bitcode interpreted, not sure, but very much doubt the source code itself is interpreted, e.g. in a loop).