elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
25 stars 438 forks source link

GCP 2.25.1 - High no. of error messages in Elastic Agent Logs #7349

Closed ar3diu closed 1 year ago

ar3diu commented 1 year ago

Elastic Agent & Stack: 8.9.0 Google Cloud Platform Integration: 2.25.1

Configuration: Elastic-Agent using the GCP integration to collect audit logs from a Pub/Sub. Issue: The audit logs actually do arrive in Elasticsearch, but there is a high no. of error messages in Elastic Agent logs, like for every audit log that's ingested. The message doesn't suggest as this would actually be about an error and I'm thinking that it could be a software bug that logs this message under a wrong log level.

{
    "log.level": "error",
    "@timestamp": "2023-08-10T12:16:48.214Z",
    "message": "ACKing pub/sub event",
    "component": {
        "binary": "filebeat",
        "dataset": "elastic_agent.filebeat",
        "id": "gcp-pubsub-default",
        "type": "gcp-pubsub"
    },
    "log": {
        "source": "gcp-pubsub-default"
    },
    "log.logger": "gcp.pubsub",
    "log.origin": {
        "file.line": 150,
        "file.name": "gcppubsub/input.go"
    },
    "service.name": "filebeat",
    "pubsub_project": "<redacted>",
    "pubsub_topic": "<redacted>",
    "pubsub_subscription": {
        "Create": false,
        "MaxOutstandingMessages": 1000,
        "Name": "<redacted>",
        "NumGoroutines": 1
    },
    "ecs.version": "1.6.0",
    "ecs.version": "1.6.0"
}
galdor commented 1 year ago

I'm seeing the same behaviour with Filebeat 0.8.9. Does anyone know what it means and how to make it stop?

elasticmachine commented 1 year ago

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

andrewkroh commented 1 year ago

I think this relates to https://github.com/elastic/beats/pull/36296 which was fixed and backported into the 8.9 branch so that it can be released when v8.9.2 comes out.

ebeahan commented 1 year ago

Fixed in https://github.com/elastic/integrations/issues/7349.