Why my Lumen is so much slower than Taylor's?

[kenjis]

According to http://taylorotwell.com/how-lumen-is-benchmarked/ :

framework	requests per second
Slim 2	1700-1800
Lumen	1800-1900
Slim 3	1200-1300

• Lumen is a bit faster than Slim2.

But my benchmarks (2015/04/15) :

framework	requests per second
slim-2.6	774.31
lumen-5.0	372.07

• Slim2 is about two times faster than Lumen.

I've found some differences in code, but overall benchmarking methodology seems to be the same.

don’t be surprised if your numbers are different than my numbers.

Yes, benchmarking environments change numbers. But the difference (relatively to Slim2) is too much, isn't it?

I simply want to know why. What makes the huge difference?

[FractalizeR]

@defenestrator

If you need extreme high-speed, web-scale, non-blocking, deeply concurrent, fault-tolerant, highly portable rock-star tech

You mean like Facebook and Wikipedia, right? :)

[neomerx]

@defenestrator If you do not have passion/time/whatever for optimizing applications it's not the reason to call those who have 'intellectual wankers'.

[defenestrator]

I am clearly misunderstood, in this case, I apologize. I'm the first to call my self an avid intellectual wanker, for the record, that's not a judgment from me. Facebook is hardly vanilla PHP. @ FractalizeR, the sentence you quoted is a little joke; https://www.youtube.com/watch?v=bzkRVzciAZg

I guess that is sort of my point. Lighten up. Yo, samdark, you're right, hard not to notice PHP is everywhere. But we don't use PHP because it is performant, we use it because it is easy to write and deploy, partially due to its ubiquity. PHP is my main language, but I'm not a zealot for anything. I took the Oath of Non-Allegiance, and I take it seriously.

[samdark]

@defenestrator PHP 7 is going to be as fast as HHVM (which, in its behavior, is compatible with Zend PHP). Yahoo uses vanilla one, others most probably do as well. PHP is, in fact, performant compared to many other interpreted languages.

[christoferd]

FYI - Recently installed a php crm on Digital Ocean the cloud server, it uses SSD, and install APC (~Zend Opcache). It's very fast, much faster than a shared hosting server. New pages loading in less than 1 second. Thought I'd share this as it may help you get more speed from app without having to change frameworks.

PS: Facebook have a developer blog, they release a lot of awesome insider dev info. Right, they don't use only vanilla php, a lot of their code is converted to straight C/C++ and turned into DLL php extension libraries. That way the library code is static, loaded once, and runs super fast. (don't quote me, it's something like that, find the exact info on their blog) I believe the Phalcon framework http://phalconphp.com does something like this also.

• Chris

On 18 May 2015 at 01:01, Alexander Makarov notifications@github.com wrote:

@defenestrator https://github.com/defenestrator PHP 7 is going to be as fast as HHVM (which, in its behavior, is compatible with Zend PHP). Yahoo uses vanilla one, others most probably do as well. PHP is, in fact, performant compared to many other interpreted languages.

— Reply to this email directly or view it on GitHub https://github.com/kenjis/php-framework-benchmark/issues/13#issuecomment-102812191 .

[kenjis]

Lumen 5.1 got performance improvement than 5.0:

framework	requests per second	relative	peak memory	relative
lumen-5.1	420.06	1.2	1.00	1.0
lumen-5.0	349.87	1.0	1.25	1.3

And slim, lumen and sliex:

framework	requests per second	relative	peak memory	relative
slim-3.0	555.19	1.6	0.75	1.0
lumen-5.1	427.53	1.2	1.00	1.3
silex-1.3	350.83	1.0	1.00	1.3

[samdark]

Hmm... are you sure you haven't updated test environment?

[kenjis]

I have updated the VM (CentOS 6) with yum update. So CentOS is now version 6.6 and PHP is 5.5.30.

[koriym]

+1 for using DO and latest stable version of PHP (5.6)

[samdark]

That's why it's faster. Not because of new framework versions.

[barryvdh]

Well, if both 5.0 and 5.1 are run on the new versions, 5.1 it's still faster ;)

[samdark]

Oh. Old ones are from new environment as well? If yes then I'm wrong about environment and indeed 5.1 improved in this regard.

[kenjis]

@koriym What is DO?

[kenjis]

The numbers are the results on exactly the same VM and yesterday. I ran the command like below:

$ sh benchmark.sh lumen-5.1/ lumen-5.0/

The difference is very big. So 5.1 is apparently faster than 5.0.

[samdark]

@kenjis DO means DigitalOcean.

[samdark]

OK. I just thought 5.0 numbers are from old run.

[OzanKurt]

So at the end is it real that slim is faster than lumen?

[trentramseyer]

9 times out of 10, yes Slim will be faster. Especially if you are building an app that other people would consume on their boxes, because every "speed test" defending Laravel and/or Lumen has discussion like this on how exactly you have to set up a box to make it comparable.

[OzanKurt]

Thanks a lot for the information. :)

[Hammad51]

here is mine, have a look

[mruz]

@kenjis look at my benchmark on the new real machine https://github.com/ice/framework/issues/95#issuecomment-204328130.

[neomerx]

@mruz For laravel you got 148 RPS but @kenjis got 427.53. And you call it 'new real machine'? Sorry you were sold :hankey: :trollface:

[mruz]

@neomerx, @kenjis uses OPcache and VM, but I don't.

There is no special settings/optimizations to boost performance of some frameworks and no cache!

[neomerx]

@mruz You will understand what you are doing when someone would cook numbers in favor of Phalcon against your Ice.

[fesor]

@mruz why don't you use OPcache? I don't know any "real" production machine which doesn't have opcache (php5.5+). It doesn't related to framework itself, just to php.

The only reason to not use opcache - to make Phalcon looks better.

Never trust a benchmark you didn't fake yourself.

For example I could take advantage of Ayres, or use php-pm with symfony/silex/laravel/lumen and get relatively the same RPS.

[neomerx]

The only reason to not use opcache - to make Phalcon looks better.

Cmon @fesor he is an honest men. His 'new real machine' just didn't have enough disk space :laughing:

[mruz]

It doesn't related to framework itself, just to php.

The only reason to not use opcache - to make Phalcon looks better.

You said that OPcache doesn't related to framework, so is it to make non Phalcon looks better?

[cebe]

@mruz to gain more insight on the effects I suggest you try it. I'd like to see how it changes things.

[mruz]

@cebe ok, I'll run it when I get back from work.

[mruz]

with Zend OPcache v7.0.6-dev:

framework	requests per second	relative	peak memory	relative	files	relative
phalcon-2.0	12,437	19.51	0.27	1.03	5	1.25
ice-1.1	11,998	18.82	0.26	0.99	4	1.00
tipsy-0.10	13,754	21.57	0.32	1.22	18	4.50
fatfree-3.5	10,957	17.19	0.42	1.60	9	2.25
slim-2.6	9,171	14.38	0.48	1.83	24	6.00
ci-3.0	7,575	11.88	0.43	1.64	26	6.50
yii-2.0	4,408	6.91	1.36	5.17	49	12.25
silex-1.3	4,407	6.91	0.80	3.04	64	16.00
fuel-1.8-dev	3,500	5.49	0.64	2.43	45	11.25
cake-3.1	2,509	3.94	1.34	5.10	84	21.00
symfony-3.0	925	1.45	2.93	11.14	214	53.50
laravel-5.1	638	1.00	2.71	10.31	39	9.75
zf-2.5	679	1.07	3.24	12.32	204	51.00