influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.57k stars 5.56k forks source link

inputs.win_eventlog does not convert (any/some?) fields to tags #15958

Open steenstra opened 2 weeks ago

steenstra commented 2 weeks ago

Relevant telegraf.conf

## Event fields to include as tags
## The values below are included by default.
## Globbing supported (e.g. "Level*" matches both "Level" and "LevelText")
# event_tags = ["Source", "EventID", "Level", "LevelText", "Task", "TaskText", "Opcode", "OpcodeText", "Keywords", "Channel", "Computer"]

# Windows Event Log Monitoring
[[inputs.win_eventlog]]
  xpath_query = '''
  <QueryList>
    <Query Id="0" Path="Application">
      <Select Path="Application">*[System[(Level &lt; 4)]]</Select>
    </Query>
    <Query Id="1" Path="Security">
      <Select Path="Security">*</Select>
      <Suppress Path="Security">*[System[(EventID = 4624 or EventID = 4627 or EventID = 4634 or EventID = 4648 or EventID = 4672 or EventID = 4662 or EventID = 4768 or EventID = 4776 or EventID = 4769 or EventID = 5379)]]</Suppress>
    </Query>
    <Query Id="2" Path="System">
      <Select Path="System">*[System[(Level &lt; 4)]]</Select>
    </Query>
    <Query Id="3" Path="Microsoft-Windows-NTLM/Operational">
      <Select Path="Microsoft-Windows-NTLM/Operational">*[System[(EventID = 8001 or EventID = 8002 or EventID = 8003 or EventID = 8004)]]</Select>
    </Query>
  </QueryList>
  '''

  only_first_line_of_message = true
  timestamp_from_event = true
  event_tags = ["Source", "EventID", "Level", "LevelText", "Task", "TaskText", "Opcode", "OpcodeText", "Keywords", "Channel", "Computer", "*WorkstationName*", "*TargetUserName*"]

Logs from Telegraf

2024-10-01T10:21:32Z I! Loaded inputs: exec win_eventlog win_perf_counters
2024-10-01T10:21:32Z I! Loaded aggregators: 
2024-10-01T10:21:32Z I! Loaded processors: 
2024-10-01T10:21:32Z W! Outputs are not used in testing mode!
2024-10-01T10:21:32Z I! Tags enabled: host=MS04
2024-10-01T10:21:32Z D! [agent] Initializing plugins
2024-10-01T10:21:32Z D! [agent] Starting service inputs
2024-10-01T10:21:32Z D! [inputs.win_eventlog] Subscription handle id:1
2024-10-01T10:21:36Z D! [agent] Stopping service inputs
2024-10-01T10:21:36Z D! [agent] Input channel closed
2024-10-01T10:21:36Z D! [agent] Stopped Successfully

System info

Telegraf v1.32.0, Windows Server 2022 21H2

Docker

No response

Steps to reproduce

  1. Edit telegraf conf to include "*WorkstationName*", "*TargetUserName*" for event_tags
  2. Start Telegraf
  3. Go to Grafana to group by a tag.
  4. Notice there is no tag for the fields Data_TargetUserName and Data_WorkstationName ...

Expected behavior

I'd expect there to be tags for Data_TargetUserName and Data_WorkstationName

Actual behavior

but there’s only fields, no tags. See screenshot. image

Additional info

No response

srebhan commented 1 week ago

Next step: reproduce issue locally.

srebhan commented 5 days ago

@steenstra sorry for the late feedback. I check the code and the issue is that the event_tags option currently does not accept wildcards... Let me see if I can add this...

steenstra commented 5 days ago

That's great to know. Do you know what the tag names are that I should use in the meantime?

Is it: Data_TargetUserName or TargetUserName

srebhan commented 5 days ago

You need to specify the full name IIRC...

srebhan commented 5 days ago

Correction. I found the issue. All the Data_* fields are coming from the XML in the event and where handled differently. I do have a PR (almost) ready and will push in some minutes. I'll link it here for you to test...

srebhan commented 5 days ago

@steenstra please test the binary in PR #16008, available as soon as CI finished the tests, and let me know if this fixes the issue!