veonik / php-router-benchmark

Benchmarks of different routing solutions
MIT License
50 stars 11 forks source link

Add a test case for the dumped Symfony2 Router. #2

Closed realityking closed 10 years ago

realityking commented 10 years ago

I've added an additional test case for Symfony2 where the routes are dumped, like it should be done in a production environment.

Interestingly, with this change the Symfony2 Router is the fastest on my machine. I'll be interested in the results with the Pux extension.

I didn't update the Readme.md, I'll leave that to you. For reference here are my results (5.4.17).

Unknown route

Test Name Time +Interval Change
Symfony2 Dumped 0.0000399455 +0.0000000000 baseline
FastRoute 0.0002344939 +0.0001945484 487% slower
Symfony2 0.0010168788 +0.0009769332 2446% slower
Pux PHP 0.0012545040 +0.0012145585 3041% slower

Last route

Test Name Time + Interval Change
Symfony2 Dumped 0.0001198003 +0.0000000000 baseline
FastRoute 0.0002490872 +0.0001292869 108% slower
Symfony2 0.0011074704 +0.0009876701 824% slower
Pux PHP 0.0014067766 +0.0012869763 1074% slower

First route

Test Name Time + Interval Change
FastRoute 0.0000158739 +0.0000000000 baseline
Pux PHP 0.0000176847 +0.0000018108 11% slower
Symfony2 Dumped 0.0000331372 +0.0000172633 109% slower
Symfony2 0.0001584002 +0.0001425263 898% slower
realityking commented 10 years ago

Note that the whole benchmark ignores another aspects of routing all together. Usually you only match once during a request (unless you support subrequests like HttpKernel does), however adding a thousand routes or so also costs time. Maybe an idea for another benchmark.

tyler-sommer commented 10 years ago

Thanks for this and the MarkdownPrinter!

It is interesting that Symfony2's dump performs so much better than the same generated array. There must be some optimizations happening when the routes are dumped which greatly improves the matching performance.

That is a good idea about another benchmark, though most of the time in production, you would only create the routes one time when warming the cache (as Symfony behaves) and all these routers offer the ability to dump routes in some cache to avoid the generation step for each request.

realityking commented 10 years ago

If you look at Symfony's UrlMatcher you'll see that Routes are actually compiled before tested. It also does some other shortcuts with the dumped matcher.

nikic commented 10 years ago

The tests I implemented for the FastRoute / Pux comparison were targeted at those particular implementations. They are not suitable for testing the Symfony router, because they implement a case where all routes have a distinct prefix and as such a prefix optimization will make the match trivial. You should try inverting the order, i.e. having the placeholders first and the distinct prefix afterwards ;)

realityking commented 10 years ago

@nikic I'm happy to entertain that thought.

Test Name Time + Interval Change
FastRoute - unknown route 0.0005982250 +0.0000000000 baseline
Pux PHP - unknown route 0.0013277069 +0.0007294819 122% slower
Symfony2 Dumped - unknown route 0.0016141894 +0.0010159644 170% slower
Pux PHP - last route 0.0017146870 +0.0011164620 187% slower
Symfony2 Dumped - last route 0.0018153846 +0.0012171596 203% slower
FastRoute - last route 0.0019525889 +0.0013543639 226% slower
Symfony2 - unknown route 0.0024326184 +0.0018343934 307% slower
Symfony2 - last route 0.0026019770 +0.0020037520 335% slower

Picture obviously changed, but differences are much less dramatic. Wonder if we can find some sort of "fair" mix, that more like a real world use case.

tyler-sommer commented 10 years ago

I think if not, completely randomizing the routes would be acceptable. A completely random prefix and postfix with the arguments in the middle should be suitable, yeah?

tyler-sommer commented 10 years ago

Though my results are still more dramatic:

Test Name Time + Interval Change
FastRoute - unknown route (1000 routes) 0.0007650691 +0.0000000000 baseline
Pux PHP - last route (1000 routes) 0.0010497823 +0.0002847132 37% slower
Pux PHP - unknown route (1000 routes) 0.0011996600 +0.0004345909 57% slower
Symfony2 Dumped - unknown route (1000 routes) 0.0018008411 +0.0010357720 135% slower
Symfony2 Dumped - last route (1000 routes) 0.0018353409 +0.0010702717 140% slower
FastRoute - last route (1000 routes) 0.0020316535 +0.0012665844 166% slower
Symfony2 - unknown route (1000 routes) 0.0072214070 +0.0064563379 844% slower
Symfony2 - last route (1000 routes) 0.0076090708 +0.0068440017 895% slower
Aura v2 - last route (1000 routes) 0.0094455910 +0.0086805218 1135% slower
Aura v2 - unknown route (1000 routes) 0.0097489083 +0.0089838392 1174% slower

Maybe I screwed up switching things? Check out my latest commit: https://github.com/tyler-sommer/php-router-benchmark/commit/d23c8ba40137946e16239231fff64fbc5872de79

realityking commented 10 years ago

That's essentially the same. I always group my routes by prefix anyway, because it's just more maintainable. But I obviously don't usually have groups as large as those currently created by the benchmark.