[meta] List of sites that give Lighthouse trouble

patrickhulce commented 7 years ago

To Investigate

[ ] https://h5.m.taobao.com/trip/home/index.html, from #3838, "no DCL in trace", "no screenshots in trace" 100% reproducible

Axe failures

[ ] http://visiondirect.com (from GTech) 🐌 🐌 🐌 invalid selector, tracked by https://github.com/dequelabs/axe-core/issues/563

Service worker/offline issues

[ ] https://www.lenskart.com/ tracked by #3724
[ ] https://next.weekplan.net
[ ] https://www.flipkart.com from #3736

Stopped tracing too early

[ ] https://www.instagram.com/p/BNQ98s4BTEa/ (from comment) tracked by #3734
[ ] https://weather.com (from comment)
[ ] https://www.davidemaggio.it/ (from comment)
[ ] http://lampsplus.com (from GTech)
[ ] http://valuecityfurniture.com (from GTech)

Site misbehavior (aborted request, failed document, extremely slow server, etc)

[ ] https://app.jalantikus.com/ (from comment)
[ ] http://createspace.com (from GTech)
[ ] http://gifts.com (from GTech)
[ ] https://music.amazon.com/ (from CDS) tracked by #3735

Working but needed more than 30s

[x] http://mcdcc.org/ 😢
[x] http://pwa.voot.com/ (from comment) 💨 🐌
[x] http://m.aliexpress.com (from comment) needs ~60+ seconds!
[x] https://nytimes.com (from comment) 💨 🐌
[x] http://hollisterco.com (from GTech) 🐌
[x] http://petfooddirect.com (from GTech) 🐌 needs ~60+ seconds!
[x] http://potterybarnkids.com (from GTech) 🐌
[x] http://rei.com (from GTech) 🐌
[x] http://shopjustice.com (from GTech) 🐌
[x] http://soma.com (from GTech) 🐌
[x] http://sportsauthority.com (from GTech)
[x] http://sterlingjewelers.com (from GTech)
[x] http://tailgateclothing.com (from GTech)
[x] http://thehappinessplanner.com (from GTech) 🐌 needs ~60 seconds!
[x] http://toddsnyder.com (from GTech)
[x] http://victoriassecret.com (from GTech)
[x] http://zulily.com (from GTech)
[x] http://www.gmarket.co.kr/ 🐌 🐌 🐌 🐌 axe takes ~30s, needs 60+ seconds

Working as of 2.6

[x] https://m.forbes.com (from #2254) 💨
[x] https://q-citrix.salesnext.com/login (from #2265)
[x] http://veerle.duoh.com/ (from #2170) (#2296)
[x] https://airhorner.com (from comment)
[x] https://react-hn.appspot.com/ (from comment) 💨
[x] https://www.nasa.gov (from comment)
[x] http://contactsdirect.com (from GTech)
[x] http://skinstore.com (from GTech)
[x] http://sunglasshut.com (from GTech)
[x] http://theshadestore.com (from GTech)
[x] http://trunkclub.com (from GTech)
[x] http://walgreens.com (from GTech) 🐌 🐌
[x] http://weddingtonway.com (from GTech)
[x] http://whitehouseblackmarket.com (from GTech) 🐌
[x] http://toysrus.com (from GTech) 💨 🐌 🐌 🐌 axe can fail

Legend

💨 = can be flaky (received a bad trace or axe failure on at least one run, but not reproducible) 😢 = undesirable metric result (but correct according to definition) 🐌 = Lighthouse is slow, i.e. multiple seconds spent in gathering/auditing

paulirish commented 7 years ago

[ ] https://app.jalantikus.com/ didn't finish its load once or twice

perf diagnostics

[ ] https://www.instagram.com/p/BNQ98s4BTEa/ is surprisingly slow to load. are we identifying the top perf concern?
[ ] http://pwa.voot.com/ is v slow to load. are we identifying the top perf conern?

paulirish commented 7 years ago

classics

[ ] https://airhorner.com
[ ] http://m.aliexpress.com
[ ] about:blank (yes, someone tried it yesterday and expected it to be fine.)

brendankenny commented 7 years ago

[ ] https://react-hn.appspot.com/ we stop too soon and only get TTCI ~50% of the time, TTFI ~80-90% of the time. Has an unusual SSR/delayed hydration load scheme that may be tricking us

brendankenny commented 7 years ago

[x] https://www.vevo.com/ may fall into the same timing oddness category as react-hn where we're cutting off too early. In my latest plots/ run, it usually had end-of-trace at 20s, but neither TTCI or TTFI were found. There were four trace outliers, however, where it went for the full 30s timeout, and for those, TTFI was found in all four, TTCI was in three. (#2271)

30s may be too short

all these struggled with TTI, but all also hit the 30s timeout

[ ] https://nytimes.com was fine with TTFI but struggled with TTCI. I assume new nytimes failures are from throttling changes from #2238
[ ] https://weather.com/ fine with TTFI but never reached TTCI
[ ] https://www.nasa.gov/ Found TTFI 1/6 of the time, never reached TTCI. onload (loadEventEnd) was usually not found in the trace taken

patrickhulce commented 7 years ago

30s may be too short

the new throttling is going to make this true for a lot of sites now, we will basically fail on all sites from deep's TTI variability study that had TTI of >25s, which is a lot

patrickhulce commented 7 years ago

re: https://airhorner.com I'm not seeing anything fishy, what fails usually? re: https://react-hn.appspot.com/ to provide more color here, this app actually renders in response to websocket messages and fixing would require a CPU quietness signal over the protocol since we can't reasonably guess when to stop tracing with the current network request signals alone

patrickhulce commented 7 years ago

[x] ~~https://plustimer.firebaseapp.com/ (fails the perf metrics with an atob is not defined debug string)~~ (#2300)

benschwarz commented 7 years ago

[ ] Bloomberg was having trouble too

patrickhulce commented 7 years ago

@benschwarz could you elaborate with what issues you're seeing there? On a couple runs the only issue I'm seeing is running up to the 30s timeout before TTCI is computed

benschwarz commented 7 years ago

@benschwarz could you elaborate with what issues you're seeing there? On a couple runs the only issue I'm seeing is running up to the 30s timeout before TTCI is computed

Ok, if it runs at all it must've been fixed in the last two weeks. 👍 The 30s timeout is a whole other thing, which is being tracked elsewhere, right?

ebidel commented 7 years ago

[ ] https://www.davidemaggio.it/ fails to finish.
large number of network requests: ~300
huge number of DOM nodes: 11,180 to be precise. Lighthouse spends a lot time in evaluating getEventListeners for ExternalAnchorsUseRelNoopenerAudit and other related audits.

upugo-dev commented 5 years ago

@patrickhulce any update on these issues? we are experiencing TTI issues quite frequently during our testing

patrickhulce commented 5 years ago

@upugo-dev these were mostly addressed by 3.0 and is quite old, probably a candidate for closing. If you have a specific, reproducible error on a particular URL then file it in a separate issue 👍

brendankenny commented 5 years ago

these were mostly addressed by 3.0 and is quite old, probably a candidate for closing

closing! We can have other issues for sites that give Lighthouse trouble without confusing things mixing in pre-3.0 issues :)

GoogleChrome / lighthouse