Reverse the relative throughput results

[motin]

It makes little sense to use relative numbers where 1.0 represents the worst framework tested. Instead, the best case (either the fastest framework or preferably no framework at all, see https://github.com/kenjis/php-framework-benchmark/issues/34) should be the base of comparison. We can use 100 as the base number for the best case.

Here is an example result that clearly shows how comparing to worst case scenario is non-informative:

|framework          |requests per second|relative|peak memory|relative|
|-------------------|------------------:|-------:|----------:|-------:|
|phalcon-2.0        |             975.53| 5,419.6|       0.27|     1.0|
|ice-1.0            |             611.45| 3,396.9|       0.26|     1.0|
|tipsy-0.10         |             752.09| 4,178.3|       0.32|     1.2|
|fatfree-3.5        |             363.96| 2,022.0|       0.42|     1.6|
|slim-2.6           |             578.86| 3,215.9|       0.48|     1.8|
|ci-3.0             |              67.70|   376.1|       0.43|     1.6|
|nofuss-1.2         |             226.13| 1,256.3|       0.59|     2.2|
|slim-3.0           |             386.59| 2,147.7|       0.62|     2.4|
|bear-1.0           |               0.20|     1.1|       0.76|     2.9|
|lumen-5.1          |             178.44|   991.3|       0.00|     0.0|
|ze-1.0             |             136.61|   758.9|       0.79|     3.0|
|radar-1.0-dev      |             153.33|   851.8|       0.71|     2.7|
|yii-2.0            |             252.91| 1,405.1|       1.34|     5.1|
|silex-1.3          |             327.53| 1,819.6|       0.00|     0.0|
|cygnite-1.3        |              72.30|   401.7|       0.74|     2.8|
|fuel-1.8-dev       |              21.44|   119.1|       0.70|     2.7|
|phpixie-3.2        |              24.92|   138.4|       1.27|     4.8|
|aura-2.0           |             149.58|   831.0|       0.89|     3.4|
|cake-3.1           |             166.20|   923.3|       0.00|     0.0|
|symfony-2.7        |              51.79|   287.7|       0.00|     0.0|
|laravel-5.1        |              76.21|   423.4|       0.00|     0.0|
|zf-2.5             |              32.31|   179.5|       2.93|    11.1|
|typo3f-3.0         |               0.18|     1.0|       0.00|     0.0|

A reversed version is more informative imo:

|framework          |requests per second|relative|peak memory|relative|
|-------------------|------------------:|-------:|----------:|-------:|
|no-framework       |           1,304.82|   100.0|       0.22|     1.0|
|phalcon-2.0        |             685.55|    52.5|       0.27|     1.2|
|ice-1.0            |             630.68|    48.3|       0.26|     1.2|
|tipsy-0.10         |             774.88|    59.4|       0.32|     1.4|
|fatfree-3.5        |             447.52|    34.3|       0.43|     1.9|
|slim-2.6           |             627.34|    48.1|       0.48|     2.1|
|ci-3.0             |             101.57|     7.8|       0.43|     1.9|
|nofuss-1.2         |             216.13|    16.6|       0.59|     2.6|
|slim-3.0           |             441.98|    33.9|       0.62|     2.8|
|bear-1.0           |              42.33|     3.2|       0.77|     3.4|
|lumen-5.1          |               0.00|     0.0|       0.00|     0.0|
|ze-1.0             |             259.65|    19.9|       0.80|     3.6|
|radar-1.0-dev      |             249.12|    19.1|       0.71|     3.2|
|yii-2.0            |             259.94|    19.9|       1.36|     6.0|
|silex-1.3          |               0.00|     0.0|       0.00|     0.0|
|cygnite-1.3        |             116.15|     8.9|       0.76|     3.4|
|fuel-1.8-dev       |              55.94|     4.3|       0.71|     3.2|
|phpixie-3.2        |              77.97|     6.0|       1.30|     5.8|
|aura-2.0           |             121.80|     9.3|       0.90|     4.0|
|cake-3.1           |               0.00|     0.0|       0.00|     0.0|
|symfony-2.7        |               0.00|     0.0|       0.00|     0.0|
|laravel-5.1        |               0.00|     0.0|       0.00|     0.0|
|zf-2.5             |              18.91|     1.4|       3.00|    13.3|
|typo3f-3.0         |               0.00|     0.0|       0.00|     0.0|

[motin]

Please re-open since the issue is not fixed yet. If closing as won't fix, please give some explanation as to why the relative numbers for requests per second are most informative when compared to whatever happens to be the worst performing framework in the test as opposed to the best performing framework. Thanks.

[kenjis]

I don't understand why the best case should be the base of comparison.

In the results, I want to know how many times faster than the worst framework. And I want to know how many times a framework use memory.

[motin]

In the result, I want to know how many times faster than the worst framework.

Why is the worst framework relevant? Why shouldn't the default comparison be against the best framework?

Check example of the output in the original post. The worst framework was typo3f, and the best frameworks were over 5000 times faster than typo3f, but for me as a PHP-developer, I couldn't care less about comparing against typo3f or some other slow framework, I want to compare against the best options out there.

Comparing to the worst framework in the world may be interesting as trivia for researchers of php frameworks, but not when the target is to understand how the fastest frameworks compare to each other.

I don't understand why the best case should be the base of comparison.

Here is a scenario: Joe the PHP developer uses Yii 2 and has grown a bit tired of it. Thinking about switching frameworks, one factor to consider is the performance. Maybe it is worth considering one of the fastest frameworks? Or no framework at all?

Joe goes and checks benchmark results, and is interested in relative performance of the fastest frameworks. When doing this, he encounters that all relative throughput figures are relative to Typo 3, which is completely irrelevant for Joe, since he is interested in the top fastest frameworks only.

And I want to know how many times a framework use memory.

Sure, this issue however only regards the relative throughput results, and not the relative-column regarding memory.