runWarnings format difficult for pragmatic usage

[benschwarz]

I have two concerns with runWarnings specifically around pragmatic usage:

• runWarnings contains both errors (eg, page hung / non-successful request on main document resource) and warnings (eg, CPU throttling might be bad)
• The errors and warnings contained within runWarnings are strings. There's no handle to be able to identify one warning/error from another, without searching the string.

Feature request summary

runWarnings to be changed to an object. It should contain code and message. Just like runtimeError.

What is the motivation or use case for changing this?

Node.js/API/PageSpeed users of Lighthouse will be able to handle errors/warnings reliably.

[paulirish]

We're thinking we can add a runtimeWarnings field on the LHR which would contain error/code combos like you described. We'd keep runWarnings around until we remove it in a major release/breaking change.

sgty?

[benschwarz]

@paulirish that sounds good. On a bit of review we also noted that audits also do not have structured error messages:

"audits": {
    "metrics": {
      "id": "metrics",
      "title": "Metrics",
      "description": "Collects all available metrics.",
      "score": null,
      "scoreDisplayMode": "error",
      "rawValue": null,
      "errorMessage": "Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_TRACING_STARTED)"
    }
  }

What do you think about expanding this issue to add runtimeWarnings to audits?

[patrickhulce]

What do you think about expanding this issue to add runtimeWarnings to audits?

It's a much larger scope to formalize every error that can be thrown in every audit. There is a small class of page-level warnings that are similar in spirit to runtimeError which makes sense to formalize.

Is it mostly the errors that are already known you'd like as an object in errorMessage?

[benschwarz]

Is it mostly the errors that are already known you'd like as an object in errorMessage?

Yep. At the moment they need to be parsed to be machine readable. Ideally every existing error message with a code would have a structured object:

{
  "code": "NO_TRACING_STARTED",
  "message": "Something went wrong with recording the trace over your page load. Please run Lighthouse again.",
}