elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
26 stars 439 forks source link

[arista_ngfw,windows] Replace legacy visualizations with Lens #10578

Closed angorayc closed 3 months ago

angorayc commented 3 months ago

Integration Name

Beat [beat] - All kinds of Beats

Integration Version

all

Agent Version

all

Agent Output Type

elasticsearch

Elasticsearch Version

*

OS Version and Architecture

*

Software/API Version

No response

Error Message

Visualizations installed with Beats are out dated. Their types are visualization, should be migrated to lens. Visualization type is not compatible in Serverless Kibana.

Please find here to view the error: https://github.com/elastic/kibana/issues/169810

We should change all the visualizations installed with Beats to lens. For example, the visualizations here should be re-generated and created in lens. https://github.com/elastic/integrations/tree/main/packages/windows/kibana/visualization

Event Original

No response

Anything else?

No response

angorayc commented 3 months ago

https://github.com/user-attachments/assets/212db286-4406-46f9-b0c2-98567bee989c

angorayc commented 3 months ago

export (6) copy.txt

The above file is created from https://github.com/elastic/integrations/blob/main/packages/windows/kibana/visualization/windows-1eeaaf70-9f23-11ea-bef1-95118e62a7c1.json

  1. Please change the extension to .ndjson
  2. Go to stack management > Saved object management > import, select the file above
  3. Open the imported file, and click Edit visualization in Lens.
  4. Save the visualization, back to Saved object management, click inspect, and then you should have the converted saved object.
andrewkroh commented 3 months ago

@angorayc I think you describing a problem with Elastic Beats (elastic/beats) and Fleet integrations. The "Average Uptime [Auditbeat System] ECS" visualization is from https://github.com/elastic/beats/blob/c86330a88d6aba199863ad250fcbea74671d01bc/x-pack/auditbeat/module/system/_meta/kibana/7/visualization/4d0485f0-18da-11e9-9094-c50574723088-ecs.json#L41. I suggest opening an issue in elastic/beats to cover those if that was your intent.

AFAIK, fleet integrations were previously migrated to use Lens (security owned packages were tracked in https://github.com/elastic/integrations/issues/6787). But according to [visualization analytics](https://visualization-usage-analytics.kb.us-central1.gcp.cloud.es.io:9243/app/dashboards#/view/c89329a0-981b-11ed-9c67-b36e35159418?_g=()) there are still non-Lens visualizations present in Fleet packages (if I'm holding it correctly). The ones related to Security would be arista_ngfw and windows.

Screenshot 2024-07-24 at 10 48 27
angorayc commented 3 months ago

@angorayc I think you describing a problem with Elastic Beats (elastic/beats) and Fleet integrations. The "Average Uptime [Auditbeat System] ECS" visualization is from https://github.com/elastic/beats/blob/c86330a88d6aba199863ad250fcbea74671d01bc/x-pack/auditbeat/module/system/_meta/kibana/7/visualization/4d0485f0-18da-11e9-9094-c50574723088-ecs.json#L41. I suggest opening an issue in elastic/beats to cover those if that was your intent.

At the beginning, it was found from a Security Project with [windows] Engine and command started visualizaiton https://github.com/elastic/kibana/issues/169810

I tested locally with Auditbeat and found the same behaviour.

We should probably check in both elastic/integration and elastic/beats folders.

andrewkroh commented 3 months ago

The believe steps to edit a dashboard are:

  1. Install the package with elastic-package install.
  2. Make the dashboard editable. See howto-make-dashboards-editable-in-kibana.
  3. Open the dashboard.
  4. Click Edit visualization in Lens.
  5. Save the visualization.
  6. Repeat step 4-5 for each visualization.
  7. Save the dashboard.
  8. Repeat for each dashboard.
  9. Export the dashboards back into the git repo with elastic-package export dashboards.
andrewkroh commented 3 months ago

Closing in favor of