myaaghubi / PHP-Frameworks-Bench

A library to make benchmarks from PHP frameworks.
GNU General Public License v2.0
109 stars 4 forks source link

Show response length in check.sh #19

Closed joanhey closed 1 year ago

joanhey commented 1 year ago

In a benchmark all need to return the same response length.

For now it's only for information, so the dev can fix it.

PD: the length is calculated only for ASCII, that for this bench is correct

joanhey commented 1 year ago

Now it's showing a very small difference, from 34 to 32 bytes. Still they need to return the same response, but it's very similar.

But sometimes can be a big diff, and it's best to fix it.

joanhey commented 1 year ago

Seems that this small difference is from the output_data.php.

joanhey commented 1 year ago

Now all have the same response size.

Before: image

After: image

So later, we can add a warning (not an error) if the length is more than 34 bytes.

joanhey commented 1 year ago

Still we have the URL length difference, in the request.

joanhey commented 1 year ago

2 bytes seems insignificant, but in a bench exist a big difference.

These 2 bytes in pure-php will be a drop of ~1.600 req/s.

joanhey commented 1 year ago

Example of a very bad benchmark:

This image is in leaf framework repo image

The people only see the Request per second, but nobody check the response size. Also the transfer rate is x2 in Slim.

myaaghubi commented 1 year ago

no idea about slim but that is the reason to have PHP-Frameworks-Bench

joanhey commented 1 year ago

The problem is that a lot of people use a Benchmarking as Bench Marketing. And any trick is OK for them.

I'm tired to explain it:

A benchmark is not a competition, it's a really good tool to optimize the code. Joanhey

But to be useful, we need to measure the data correctly.