paulirish / pwmetrics

Progressive web metrics at your fingertipz
Apache License 2.0
1.25k stars 74 forks source link

"Something went wrong with recording the trace over your page load" error #166

Closed jooj123 closed 6 years ago

jooj123 commented 6 years ago

Config / CLI options

module.exports = {
  url: 'http://localhost:3000/blahblah',
  flags: {
    runs: '2', 
    submit: false, 
    upload: false, 
    view: false, 
    expectations: true,
    chromeFlags: [
      '--headless',
    ],
  },
  expectations: {
    ttfcp: {
      warn: '>=7189',
      error: '>=9000',
    },
    ttfmp: {
      warn: '>=8387',
      error: '>=10000',
    },
    psi: {
      warn: '>=30842',
      error: '>=35000',
    },
    fv: {
      warn: '>=7190',
      error: '>=8000',
    },
    vc: {
      warn: '>=30912',
      error: '>=35000',
    },
    vc85: {
      warn: '>=28759',
      error: '>=30000',
    },
  },
};

Steps to reproduce.

Run: pwmetrics --config=test/performance/pwmetrics-config.js as npm command

Stack trace

Errors like:

Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: first-meaningful-paint

Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: estimated-input-latency

etc for each metric

Screenshot

screen shot 2018-03-29 at 7 44 06 pm

Environment

  1. pwmetrics version: 3.1.6 3.2.1
  2. Chrome version: 64
  3. OS version: node:8.9.4-slim but I can reproduce this on my mac sometimes too

This issue means I can no longer run lighthouse in my CI setup - any ideas would be greatly appreciated

denar90 commented 6 years ago

Can you try v3.1.7? looks like this one has to be fixed in latest version

jooj123 commented 6 years ago

Updating doesnt seem to make a difference - i still get these messages

jooj123 commented 6 years ago

I have noticed this goes away if I use the --json output flag.

Seems that wunderbar is causing issues (i missed the last bit of errors in original post): " TypeError: Unexpected options.max: null. Expected number 14:54:33 [tests] at normalizeOptions "

14:54:26 [tests] Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: first-meaningful-paint
14:54:26 [tests] Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: estimated-input-latency
14:54:26 [tests] Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: first-interactive
14:54:26 [tests] Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: consistently-interactive
14:54:26 [tests]   ✓ Success: Run 1 of 2 finished successfully.
14:54:26 [tests] Launching Chrome
14:54:33 [tests] Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: first-meaningful-paint
14:54:33 [tests] Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: estimated-input-latency
14:54:33 [tests] Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: first-interactive
14:54:33 [tests] Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FMP) Audit key: consistently-interactive
14:54:33 [tests]   ✓ Success: Run 2 of 2 finished successfully.
14:54:33 [tests]         ☆  Median run  ☆
14:54:33 [tests] TypeError: Unexpected options.max: null. Expected number
14:54:33 [tests]     at normalizeOptions (/workspace/search-results_PR-1874-head-PPB4NMFRURBGLRE3KKGPQTRTPXHLZJSJNIIQ55HIBBQO44HPZKKA/node_modules/@gribnoysup/wunderbar/lib/normalizers.js:47:11)
14:54:33 [tests]     at draw (/workspace/search-results_PR-1874-head-PPB4NMFRURBGLRE3KKGPQTRTPXHLZJSJNIIQ55HIBBQO44HPZKKA/node_modules/@gribnoysup/wunderbar/lib/index.js:62:7)
14:54:33 [tests]     at drawChart (/workspace/search-results_PR-1874-head-PPB4NMFRURBGLRE3KKGPQTRTPXHLZJSJNIIQ55HIBBQO44HPZKKA/node_modules/pwmetrics/lib/chart/chart.js:14:23)
14:54:33 [tests]     at PWMetrics.showChart (/workspace/search-results_PR-1874-head-PPB4NMFRURBGLRE3KKGPQTRTPXHLZJSJNIIQ55HIBBQO44HPZKKA/node_modules/pwmetrics/lib/index.js:232:9)
14:54:33 [tests]     at PWMetrics.displayOutput (/workspace/search-results_PR-1874-head-PPB4NMFRURBGLRE3KKGPQTRTPXHLZJSJNIIQ55HIBBQO44HPZKKA/node_modules/pwmetrics/lib/index.js:216:18)

cc @gribnoysup must be associated with: https://github.com/paulirish/pwmetrics/pull/161

gribnoysup commented 6 years ago

@jooj123 Can you share the output that you get with json flag? I assume there is timing missing for some metric, but instead of the empty chart as before, you are getting library validation error. At the time of the original post, wunderbar branch wasn't merged yet 😅

I actually didn't though enough about the case where PWMetrics fails to record data. @denar90 do you think we should fail if some required data is missing or maybe a better idea would be to not print charts at all (or print an empty one as it was before, although this seems a little bit confusing)?

jooj123 commented 6 years ago

Yeah maybe its a seperate issue - because as you say the original issue didnt have wunderbar merged (I have since updated pwmetrics and get the same errors + the wunderbar ones)

Maybe the timing was missing - but if thats the case it should still be graceful about that and not fail with errors like: TypeError: Unexpected options.max: null which dont make alot of sense to devs

Its hard for me to reproduce it now, ill try again later (its a transient issue - possibly some sort of bug with lighthouse as to why it fails to generate metrics I guess)

Logged a seperate issue for this here @gribnoysup: https://github.com/paulirish/pwmetrics/issues/175

denar90 commented 6 years ago

@jooj123 feel free to reopen this one in case of bug reproducing)