treffynnon
commented
10 years ago Indeed and that is my problem with benchmarks like this - they're not representative and it is near impossible to make them so.
As I alluded to in the article I wanted something that could be easily transferred between the various runtimes using the same algorithm. Unfortunately this ultimately means that the chosen task will always be too trivial. I simply did not have enough spare time to make it any more complex - it took me long enough as it is! :)
Sara Golemon also pointed out on Twitter most of us are not using PHP on the command line either, but rather through a web server of some kind. This is, of course, true; but I wanted to eliminate any overhead that the server might add. This means that the tests are hitting startup and shutdown costs that a webserved benchmark would not encounter.
Oh and hey - pull requests/forks are welcome! :)
evert
alvinwoon
I was invited to leave any comments as issues on github, so here it is ;)
You mentioned yourself: all benchmarks are flawed.
Well I don't completely agree, if you're benchmarking the actual goal you're trying to achieve, you can get some pretty representative numbers.
Your benchmark, (while both impressive and interesting!) is, I feel, not a good way to measure performance when comparing PHP engines.
99.99% of us are not calculating mandlebrot sets. Most of us are not doing much calculations at all. My personal guess would be that most of the time is spent on in PHP are:
I'd love to see a benchmark one day that addresses this (more typical) usecase.
Still pretty cool though =)