search

edin / raytracer

Performance comparison of various compilers
MIT License
57 stars 22 forks source link

avoid measuring compilation time #10

Closed lmiq closed 3 years ago

lmiq commented 3 years ago

In Julia, the code is compiled in first call to every function, thus compilation time was being measured here, and might be significant. Running the two commands before the time-measure is more appropriate (or, alternatively, using @btime from BenchmarkTools, except I don't know if you are willing to include other packages not from the base language).

edin commented 3 years ago

Thanks, I think it's ok to use if something is idiomatic for the language and it's easy to install. Ideally it would be nice to use some external tool to measure time and to verify that image is correctly generated.

lmiq commented 3 years ago

Benchmarking quick executions of Julia codes from outside is hard because of the intrinsic compilation that occurs on that first call. Actually, the way it is now is ok for comparison purposes, because the benchmartools will report the minimum time of many runs (which is a better measure, but not what is being done for the other languages). By the way, here the Julia code runs 50% slower than the Fortran version (not 6x slower as reported in the table). Maybe the time there included the interpreter startup time?