ageitgey / amplify

A Jekyll html theme in the vague style of Medium.com built using Google AMP
https://cdn.ampproject.org/c/s/ageitgey.github.io/amplify/2016/03/08/example-post.html
Other
1.87k stars 166 forks source link

Comparing your theme with fb/react gh-pages is so irrelevant #2

Closed MoOx closed 8 years ago

MoOx commented 8 years ago

FB react contains react, and a live coding editor, why don't you choose something more relevant like a normal/simple jekyll theme?

Prinzhorn commented 8 years ago

I just compared it to my own website, which is also just a static website. It loads even faster than your demo page. It also contains an image. It also contains Google fonts. It even uses an external CSS file. And all of these are blocking. No AMP magic.

It don't want to hate on AMP, just saying that the comparison is not fair at all (in the same way that my comparison is not fair! or even a valid benchmark). I also wanted to show of how fast my website is :-D. It uses CloudFront, the numbers below for "first page visit" mean that my local cache is disabled, but my closest edge location already has the data (once you have more than five visitors a day the edge locations should always be warmed up anyway). You can have most of the AMP things without using AMP. But they did a good job of writing them down once (like a style guide), forcing these rules (Google won't let you in if it's not valid AMP) and advertising AMP.

One thing that AMP pages benefit from is that assets like amp-youtube-0.1.js can be cached across multiple pages if enough publishers join. Similar to how many sites include jQuery from the Google CDN.

First page vist with no throttling

Page DOMContentReady Load
https://www.prinzhorn.it/ 55ms 144ms
https://cdn.ampproject.org/c/s/ageitgey.github.io/amplify/2016/03/08/example-post.html 87ms 307ms

Second page visit with no throttling

Page DOMContentReady Load
https://www.prinzhorn.it/ 63ms 60ms
https://cdn.ampproject.org/c/s/ageitgey.github.io/amplify/2016/03/08/example-post.html 104ms 205ms

First page vist with "Regular 2G" throttling

Page DOMContentReady Load
https://www.prinzhorn.it/ 368ms 4.64s
https://cdn.ampproject.org/c/s/ageitgey.github.io/amplify/2016/03/08/example-post.html 570ms 6.43s

Second page visit with "Regular 2G" throttling

Page DOMContentReady Load
https://www.prinzhorn.it/ 349ms 345ms
https://cdn.ampproject.org/c/s/ageitgey.github.io/amplify/2016/03/08/example-post.html 387ms 515ms
MoOx commented 8 years ago

In this section, we clearly see facebook.github.io/react https://github.com/ageitgey/amplify#how-fast-is-this-theme

MoOx commented 8 years ago

@Prinzhorn you have 55s for domcontentloaded? Seems like a mistake.

ageitgey commented 8 years ago

@MoOx Sure, I updated the readme to also include the Jekyll homepage which is much lighter.

@Prinzhorn Yeah, Google AMP is definitely not required to make a fast page. The problem is that without a good set of rules, most people don't make fast pages. But nice job making your page load very quickly!

MoOx commented 8 years ago

Why don't you just remove react of the tests?

acdha commented 8 years ago

On a more general note, it would be good to use a real browser – e.g. not the developer tools which can disable certain features — and report the time to render, which is the most critical metric for user-perceived performance. I would recommend using WebPageTest.org since that gives you robust cross-brower tests and makes it easy for anyone to repeat the same test using the same settings.

The difference is fairly dramatic, as well — consider http://www.webpagetest.org/result/160315_YG_XGP/3/details/ using Chrome with the “Slow 3G” network profile:

domInteractive: 0.950s domContentLoaded: 950s first Paint: 2.016s loadEvent: 6.494s

Since users care when they can start reading the content, that domInteractive time is misleading by a factor of two and the filmstrip view shows that the actual time the user cares about is 3.5 seconds:

http://www.webpagetest.org/video/compare.php?tests=160315_YG_XGP-r%3A1-c%3A0&thumbSize=200&ival=100&end=visual

EDIT: here are more runs adding Firefox to the mix:

http://www.webpagetest.org/video/compare.php?tests=160315_TD_XVX,160315_MT_XVR,160315_7T_XSQ,160315_H2_XQN,160315_YG_XGP

ageitgey commented 8 years ago

Great idea! I'll update the README with a webpagetest comparison.