[Lens] Show warning messages when switching to an index pattern with incompatible fields

[flash1293]

The index pattern datasource should help the user in configuring the data source they want. This includes switching the index pattern for already configured layers. However this has some weird edge cases which should be handled correctly.

Discussion carried over from https://github.com/elastic/kibana/pull/42427 because there might be multiple PRs around that topic.

@flash1293

I'm worried that the UI could be confusing in some situations:

The user wants to create one layer with index pattern A and one layer with index pattern B. They configure the first layer and then switch to index pattern B in the data panel -> layer 1 switches to index pattern B as well and the user has to switch it back to A manually. The user creates a layer with index pattern A and switches to index pattern B on the left side. The layer changes the index pattern to B which is what the user expected. Then the user changes to index pattern C which is restricted (rollup indices), expecting the current layer to also switch to C, but that doesn't happen because the current operations are not compatible.

@AlonaNadler

The user wants to create one layer with index pattern A and one layer with index pattern B. They configure the first layer and then switch to index pattern B in the data panel -> layer 1 switches to index pattern B as well and the user has to switch it back to A manually.

How about we have a popup that says something along the lines of "Lens found the same fields in index B would you like to apply index b on the current visualization? yes change to index B / no do nothing`

The user creates a layer with index pattern A and switches to index pattern B on the left side. The layer changes the index pattern to B which is what the user expected. Then the user changes to index pattern C which is restricted (rollup indices), expecting the current layer to also switch to C, but that doesn't happen because the current operations are not compatible.

How about we have a popup that says you switched index pattern, how would you like to proceed? clear my chart/keep the chart as is and we open a new layer in that case

@flash1293

That's a nice way of handling that, but do we want to show a popup each time the user switches the index pattern (which would be implied by your suggestion above)?

How about only showing the popup if we could transfer the layer and just doing nothing if that's not the case? A new layer will be opened automatically once the user drags a field from the new index pattern into the workspace.

To make it less intrusive, @wylieconlon also had the idea to show the popup inline in the data panel (maybe above the field list).

@cchaos

I had a similar idea to @wylieconlon's but in the layer panel instead since that's where the "switch" would happen.

@cchaos Could you throw together a quick mock around this?

[flash1293]

@AlonaNadler @cchaos I created a simple version of switching between index patterns manually in here https://github.com/elastic/kibana/pull/42599:

[flash1293]

@AlonaNadler @cchaos Another suggestion for the "semi automatic switch":

If the user switches the index pattern in the data panel, this text is shown:

(The bolt icon is only shown if there are incompatible operations, otherwise it's just the text)

[cchaos]

Here are 2 mocks for you.

Changing the source in the Source panel

Changing the source in the Config panel

[flash1293]

I like the concept @cchaos - what do you think about extending that with the partial switch from two commons up? If not all operations are compatible, it throws away those which are not compatible but keeps the others. keeping as much as possible is better than nothing here IMHO

[cchaos]

You could try something like this

[flash1293]

That’s what I had in mind. @AlonaNadler @wylieconlon what do you think? Any objections against this?

[AlonaNadler]

That's a nice way of handling that, but do we want to show a popup each time the user switches the index pattern (which would be implied by your suggestion above)?

• Switching an index is not that common, I don't have a problem if it will show a popup each time users switch to a new index pattern. It's not trivial to change the underlining data in this way and it should be explicit.
• The use case of switching the index and replacing the existing index in the configuration with the new index pattern is more common and we should be optimized the UI for this use case instead of the multiple layers each with different index pattern (which opens a lot more edge cases we have not yet considered)
• switching the index pattern fields by default keeps it simple, I prefer to keep the layers and multiple index patterns to power users, we will support combining multiple index patterns on the same chart but it shouldn't be that straightforward from the UI. That is why I suggest if users have only one layer to keep the source panel index in sync with the layer, the index pattern can appear in the layer only when there are more than 1 layer.

[wylieconlon]

I think this is still in issue for beta.

[wylieconlon]

@AlonaNadler @cchaos The current state of this issue is unchanged in about a year. Do you think the current behavior is good enough that we can close the issue? The current state is that when switching to a new index pattern, any non-matching fields are removed without warning. This issue was describing how instead of showing no warning, we would want to show a warning.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[markov00]

Currently the index switch is per layer, so we are not changing/switching suddenly all the layers configuration at once. The fact that we are not warning the user still there but is a minor problem and can be reconsidered in the future is we receive user feedback from that. For now I'm closing this issue