search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
38 stars 449 forks source link

[Windows] Agent integration unable to keep up with events ingested by WEC #7938

Open Acmosa opened 1 year ago

Acmosa commented 1 year ago

We run in to an issue where the Agent is not able to keep up with the Windows Event received. We did not have this problem when we were using Winlogbeat. I would like to ask two things.

  1. Has the mechanism changed in the Elastic Agent with the Windows integration from the way Winlogbeat retreives Windows Events?
  2. Can you add a calculated field in the Windows integration pipeline that add the time difference between the @timestamp and the event.created fields before the event hits the Elastic cluster for ingestion. This way we have a numeric field we can use in an alert to see if this value increases beyond our requirements.
elasticmachine commented 4 months ago

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

marc-gr commented 4 months ago

Which version of winlogbeat were you using prior? For integrations winlogbeat processing was moved from the beats host to ingest pipelines. It could be possible that your ingest node is not handling that load properly.

Acmosa commented 4 months ago

Which version of winlogbeat were you using prior? For integrations winlogbeat processing was moved from the beats host to ingest pipelines. It could be possible that your ingest node is not handling that load properly.

I am sorry for not informing you. The issue was resolved by scaling our ingest nodes after a call with Elastic support. However I couldn't get a strait answer on the questions posed here. I think it still relevant to have these fields for easy monitoring purposes. Unless there is another way... Thanks.

intxgo commented 3 months ago

based on the solution, perhaps you'd want to monitor the difference between event.ingested and event.created instead, or both of them