search

GoogleChrome / lighthouse

Automated auditing, performance metrics, and best practices for the web.
https://developer.chrome.com/docs/lighthouse/overview/
Apache License 2.0
28.02k stars 9.32k forks source link

Add RUM data #1453

Open ebidel opened 7 years ago

ebidel commented 7 years ago

One thing I've been thinking a lot about is how to incorporate RUM data into LH reporting.

LH suggests many ideal targets for developers to hit, in particular perf numbers. Incorporating real-world data from sources like httparchive and caniuse could be a helpful barometer for devs to see where they're landing.

Where RUM data would be useful:

For httparchive data, I threw together a webservice on App Engine that always pulls from the latest snapshot of httparchive's bigquery dumps. Turns out they didn't have a JSON API.

endpoint: https://lighthouse-viewer.appspot.com/data demo: https://lighthouse-viewer.appspot.com/

I also have a node script that we could run during report generation. The benefit of the webservice is that anyone can use it and it's highly cached.

POC UI:

lighthouse_httparchive_integrated

There's a lot we could do in this space. I'm interested in hearing everyone's thoughts. Need some UI explorations. We could annotate certain audits (like the screenshot) or have an entire new section for RUM stuff.

@paulirish @igrigorik

igrigorik commented 7 years ago

I like where you're heading. Providing a baseline to help developers understand how their pages stack up is a great way to help flag outliers in particular... "oh hey, my page seems to be consuming way more image bytes, maybe I should..."

FWIW, imgix page weight tool does a pretty good job of this (e.g. sample report). In particular, I really like that they show the percentile stats ("The site as measured is in the 6th percentile for all mobile sites") and the possible win ("Optimizing images could move this site to the 5th percentile of mobile sites") if you apply their recommendations... Granted, determining the latter is not possible in many cases, but even the current percentile on its own is already helpful in providing a lot of context.

As a separate topic, one thing to be careful with is number+stats fatigue. We don't want to overwhelm the user with all the comparisons and percentiles. To address that, we could selectively highlight particular metrics (e.g. page weight), and/or only show the percentiles when certain thresholds are hit -- e.g. we've noticed that you're page is in 85th+ percentile for JS bytes, perhaps you should... and vice versa (as a footnote, perhaps): congrats, your page is in top 10% for JS bytes..

paulirish commented 6 years ago

On the roadmap for q2 2018.

mgifford commented 5 years ago

Ok, I have to admit I didn't know this one - Real User Monitoring = RUM.

My mind went elsewhere.

connorjclark commented 5 years ago

On the roadmap for q2 2018.

lol. so, where is it now :)

wardpeet commented 4 years ago

Is this still something we want to put in lighthouse locally? This feature currently lives in PSI with the Crux implementation. Making audits/result be aware of analytics will probably be a big assignment.

alekseykulikov commented 4 years ago

@wardpeet the field-performance plugin does precisely this. It adds CrUX data using PSI API to Lighthouse report https://github.com/treosh/lighthouse-plugin-field-performance

paulirish commented 4 years ago

@alekseykulikov btw as of a month ago, we have a crux-only mode for the api, though you can currently only get origin-level data.

https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=origin%3Ahttps%3A%2F%2Fdevelopers.google.com

(note the origin: in the URL)

We're talking about supporting url: in there. too.

cc @exterkamp

alekseykulikov commented 4 years ago

Thank you for adding this @paulirish! Indeed, the url: mode can be handy, and field-performance plugin will use this new API.

exterkamp commented 4 years ago

hot take: url: is GA :smile:

url vs. origin