ngx_pagespeed performance question

[ghost]

We installed ngx_pagespeed the 25th of April, removed it the 29th, installed a new build from the Master (after bug https://github.com/pagespeed/ngx_pagespeed/issues/330 was solved and removed it Tuesday, May the 7th).

As you can see above, the resultats are not what was expected

These figures are based on several hundreds of thousands of visits during the period of time.

What we can read is when ngx_pagespeed is on, the average page load time increases. Even more confusing is in the mean time we tested several configurations Pagespeed Off and Pagespeed On (On, Perf and Perf+ based on different filters configurations from basic to more risky).

I really don't know what to think of this. Any idea?

[patschi]

I guess the load average is higher than before, because ngx_pagespeed is compressing the JavaScript and CSS code and so the browser needs a little bit longer to "understand" and execute the compressed code.

[vidluther]

You also need to measure perceived load time, versus actual load time. Sometimes also referred to as render time.

Notice your average page download time, it's 3 seconds lower.

Actual page load time can vary because you may have javascript and images loading from domains outside your control, and if they have shitty connections, the "page load time" will be higher.

Look at this video: http://www.youtube.com/watch?v=rwmuRfB68js

Even though the # of seconds is high, the website is actually usable for the end user, a lot faster with PageSpeed.

[jeffkaufman]

The key issue here is that while in the experiment results you are seeing an improvement in time until onload ("Avg PageLoad Time") that improvement is not backed up by improvements in your overall numbers.

The "Avg Page Download" metric might be illustrative here: it measures the time for downloading just the raw html from your server. There's no reason it should be higher with PageSpeed off; if anything inlining and the computations PageSpeed needs to make should make "Avg Page Download" worse (and then PageSpeed should be able to make up for it elsewhere to manage a lower PageLoad time). Same with "Avg Server Response Time"; there's no reason it should be higher in the PageSpeed off group.

One potential issue is that the averages in GA are extremely sensitive to outliers. One user who randomly takes five hours because they put their computer to sleep in the middle of loading a page can have a huge effect on the average while not telling us anything. One way to get around this is to see if you get consistent results when you look at individual days. Would it be possible for you to give me read access to your Analytics account so I can do that, or might you be able to do that and let us know?

What period of time were you running with #330 fixed?

[ghost]

Hello Jeff, I've just invited you under GA. We had the master installed with #330 fixed from April, 30th until May, 6th. Under Advanced segments you'll see our tests.

[jeffkaufman]

We had the master installed with #330 fixed from April, 30th until May, 6th

I'm confused; I only merged the fix for #330 to master on 5/6 (8c88593be938e).

[ghost]

Sorry I meant that we installed the fix the day we discussed it (ie last Monday, the 6th in morning) and removed it tuesday (May, 7th) in the afternoon.

[jeffkaufman]

So you were running with #330 fixed from the morning of 2013-05-06 to the afternoon of 2013-05-07?

[ghost]

yes

[liluxdev]

I never found reliable google analytics load time report (not yet), it merges all the browsers load times and does a simple avg...

but what about if someday you get many vistiors with ie6?? or many visitors from china when for instance your server is only in US?

The average load time goes up randomically....

you have to get the avg of modern and serious browsers and of your targeting location!

for this try a free account on https://www.pingdom.com/rum/ that is well done and calcuates avg load times clustered by browser, conutry etc.

[ghost]

I agree with you. Since we have a pingdom account, we can corrobrate GA figures.

Peaks are related to #330. We see that after we installed the fix, we no longer encountered outages, but can see that we had longer response times from the site, and when we removed ngx_pagespeed (7th) response times improved.

[liluxdev]

Your pingdom response time report (chart n.1 ) confirms that after you upgraded to the new version of ngx_pagespeed the bug disappeared and situation improved server-side, but chart n.2 doesn't tell anything bad, it's normal that with pagespeed enabled server side repsonse time can go a bit up (it's a piece of software running on your cpu, it can bring your load average a bit up and if your server is at limit - and you don't cache enough - this can cause higher server response time but pagespeed anyway should improve user experience load time and if configured well with some cache also server load can benefit of pagespeed -less requests- but this is it's not so easy to achieve)

Anyway I was meaning RUM pingdom reports (Real User Monitoring, a new feature you have to activate and install with an async script like GA ) the one you posted now is the "wget style" report, the time to download a single page, not a real browser metric (and it confirms it was a severe server-side issue!)

Day details per browser: Day details per platform: Day details per country:

As you can see in my case the tablets with lower bandwith on an old non responsive site are inflating up the median load, but I know, it's normal, so i'm developing a responsive version of the site. (phones got lower times compared to tablet just because my site has a mobile version served using the bad and old user-agent detection method)

ModPagespeed is an awesome tool and helps a lot, but page speed optimization still depends on a lot of complex factors, so you have to mesure where is the problem to find an appropriate solution to improve your site speed.

Anyway i'm pretty sure that your users with modern browsers and desktop client have improved load time if you keep pagespeed on, try to measure it with RUM

If you cant see RUM on your pingdom account you've to create another free account with them (known issue for old accounts).