[Lens] allow rendering despite invalid dimensions/layers

[drewdaemon]

Describe the feature

Today in many cases Lens will not render a chart because of a single invalid dimension or layer. The rest of the chart configuration may be perfectly valid and if the problematic portion of the configuration were ignored, we could still show the user a partial chart while still signaling the problem with this dimension-local pattern we use for reference lines:

This would make for a better user experience than we have today since

• It doesn't throw up a hard roadblock—it's more beneficial since they can still visualize the things that are working
• More localized error messaging which is more clearly actionable because they know exactly where to make the change

We could also make the error icon mutually exclusive with the existing swatch/collapsed/palette dimension icon.

Specific use cases

• partial subsequent data layer on XY (could just exclude that layer and still render the first)
• extra bucket dimensions on partition chart
• counter rate and remove date histogram
• partially configured cumulative sum
• incorrect formula
• required dimensions (i.e. we may have a dimension marked as required, but technically be able to render something even when it isn't there)
• annotation layer's data view missing

Questions

• Saving behavior with invalid configuration

Related questions

• behavior with missing fields (https://github.com/elastic/kibana/issues/143673)
• what is an error, what is a warning? Does it change based on context (i.e. integration dashboard/embeddable vs user just building ad-hoc visualizations)

Technical thinking

This change would involve allowing the visualization class to mark certain dimensions as invalid for all purposes (including data fetching) except display to the user.

One idea from @dej611 could be creating a validation pipeline that is applied to the datasource and/or visualization state before that state is used for anything but displaying the dimension configurations.

const validState = getValidState(datasourceState, datasource, visualization)

// none of the invalid dimensions will be fetched because they weren't used when building the expression
const expression = datasource.toExpression(validState);

[elasticmachine]

Pinging @elastic/kibana-visualizations @elastic/kibana-visualizations-external (Team:Visualizations)

[drewdaemon]

With the right (wrong?) data view, pre-ingest integration dashboards can still look awful, even after https://github.com/elastic/kibana/pull/149262. This issue should fix this because it will prevent invalid configurations from making it to the renderer.

https://github.com/elastic/kibana/pull/149262 was basically a "let-it-fail" solution. Works okay in many scenarios, but just how many was never tracked or studied.

The following dashboard contains many visualizations using the same data view. They are all (but one) failing for lack of field data while the renderer builds the elasticsearch request (esaggs has its own set of validation checks).