search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
186 stars 391 forks source link

[GCP] New datasets should not be enabled by default. #8769

Open leandrojmp opened 6 months ago

leandrojmp commented 6 months ago

Hello,

Everytime a new dataset is added to the GCP integration it is enabled by default, since those datasets may require aditional permissions that are not enabled on the credentials being used, this will make the agent running the integrations to be unhealthy and require manual intervention.

For example, we use the GCP integration to only get GCP audit logs, a new dataset, Cloud Run Metrics, was added on version 2.31.1, and our credentials do not have permissions to this kind of data, and it also seems to need extra configurations, so this made the Agent to be Unhealthy because of this error.

It would be better for the new datasets to not be enabled by default when upgrading the integration.

andrewkroh commented 6 months ago

I agree. New datasets enabled by default creates the same problems we had with Filebeat modules (https://github.com/elastic/beats/issues/17256).

leandrojmp commented 4 months ago

Hello @andrewkroh

Just an update, this also happens on other integrations, we updated our AWS integration and one of the changes is that a new dataset/metricset for the AWS MSK was added, and this was enabled by default, since we do not use MSK and our credentials do not have permission to its metrics, the agent entered on an UNHEALTHY state.

Is there a generic issue about this?

andrewkroh commented 4 months ago

I'm not aware of a generic issue. But perhaps the Fleet team is aware of the issue and tracking it somewhere. @elastic/fleet?