Migrate visualisations to Lens

[ruflin]

Lens is the main editor that should be used for visualisations. Existing dashboards we own must be migrated to it. In https://github.com/elastic/integrations/issues/265 a previous discussion happened on Gaps to moving to Lens. The obvious ones are resolved but there might be more we uncover during the migration. This issue has the purpose to uncover these issues and track the migration to Lens. Any blockers during the migration must be mentioned in this issue. The current state of the migration from legacy visualisations to Lens for integrations can be found here) (private link).

On overview from the state of 2022-12-06 is shared on the bottom. For tracking, I will list the top 5:

• [x] #4994
• [ ] System @elastic/elastic-agent-data-plane
• [ ] Cef @elastic/security-external-integrations
• [x] #4995
• [ ] #5015
• [ ] Network Traffic @elastic/security-external-integrations

For the migration, there is an "Open in Lens" button which should hopefully help with the migration. The overall goal is to migrate all visualisations but lets start with the top 5 as the initial priority to also uncover potential migration blockers.

[ruflin]

@jsoriano Similar discussion applies here like for by reference. Can we add some hints during the build process or similar to warn devs?

[jsoriano]

Similar discussion applies here like for by reference. Can we add some hints during the build process or similar to warn devs?

We need some better way to encourage developers to use more current standards. Printing warnings as we do now in some cases is easily ignored or overlooked. Producing errors in CI is usually too disrupting and difficult to introduce. We'd need to find some intermediary approach. Maybe https://github.com/elastic/package-spec/issues/342 can help, but is probably not enough.

We may need some mechanism to encourage warnings reduction in CI, maybe we can do something like failing if a PR affecting a package with warnings doesn't reduce the number of them.

[ruflin]

@jsoriano Agree. This might become a longer discussion, lets take it to a separate issue.

[drewdaemon]

@gvnmagni is part-way through a Lens-based refactor of the system metrics dash!

[kaiyan-sheng]

Which version of Kibana starts supporting "Open in Lens" button?

[drewdaemon]

@kaiyan-sheng it was first introduced in 8.2. We have continued adding support for further configurations since then, especially in 8.5 and 8.6 when we finished all the cases we outlined for phase 1. This is the overall meta-issue that links to everything.

So, a gradually increasing number of legacy visualizations are convertible as you advance from 8.2 forward to 8.6 where the vast majority can be (still subject to human review).

In 8.7, we'll be promoting "Open in Lens" on dashboards (you can see some screenshots on https://github.com/elastic/kibana/pull/146363) which will make it more obvious which visualizations can be automatically converted and easier to convert them.

[ruflin]

One challenge we will have during converting to Lens is that many / most of the packages are backward compatible with 7.17 which means there are limits on the features that 7.17 has. At some point, the compatibility version needs to be increased of the package so we can make use of all the features in Lens.

[milanparmar-crest]

Team FYI, We're currently scoping this effort and we are planning to work on this issue.

[roshan-elastic]

We have a load of visualisations in the hosts view which would benefit from moving to Lens (I had an SRE interview today where they were expecting to be able to go into lens and add):

+1 on the effort (you can assume the Infra UI Team will prioritise this effort where it adds business value - I'm seeing this already)

[milanparmar-crest]

Created an issue to track the Observability integrations on which we are working to migrate in Lens. https://github.com/elastic/integrations/issues/5221

[kush-elastic]

We have dashboards that contain Input Controls visualizations and are now deprecated. We are thinking of using new Controls (introduced in 8.3) as part of this. We found a few discrepancies between the Input Controls visualization and Controls panels as below.

Input Controls Visualization:

Control:

Input Control visualization: We can make dependency filters that strictly depend on each other. New Controls: Dependency filtering is possible but filters are not required to be strict. In Input Control visualization, the default placeholder is Select while in new Controls it is Any. (I don't think it will affect users)

There shouldn't be any functional difference between these two panels.

[drewdaemon]

cc @ThomThomson ^^

[ThomThomson]

Hey @kush-elastic, by strict, do you mean subsequent controls being disabled until the controls that come before are selected?

If so, are any clients confused or annoyed by this change in behaviour? From our research, it seemed like clients prefer being able to select from any control in the chain. When we created the new Controls, we wanted to enhance the functionality rather than being beholden to exactly how the old Controls worked.

[kush-elastic]

@ThomThomson sorry but I guess it was confusing, I just had a doubt that if we use new Controls will it affect users in any way or not. If this new Controls were created in the manner of solving the disabling filters then i think it is right as well. We will be using new Controls to replace older input controls. Thank you 👍

[ThomThomson]

@kush-elastic, let us know if you run into any trouble, or have any feature requests or feedback for the new Controls!

[kush-elastic]

@kush-elastic, let us know if you run into any trouble, or have any feature requests or feedback for the new Controls!

@ThomThomson Sure, Thanks for the quick response.