Closed ethack closed 2 days ago
Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)
I just uncovered a few additional details.
This is giving a lot of errors for us.
In System integration 1.60.2:
{
"gsub": {
"field": "source.ip",
"pattern": "::ffff:",
"replacement": "",
"ignore_missing": true
}
}
Logstash output:
{
"response": {
"create": {
"error": {
"reason": "[1:1203] failed to parse field [source.ip] of type [ip] in document with id 'KLzoLJEBQHuHIPLF3fpq'. Preview of field's value: 'fe80ffff:ffff:ffff'",
"caused_by": {
"reason": "'fe80ffff:ffff:ffff' is not an IP string literal.",
"type": "illegal_argument_exception"
},
"type": "document_parsing_exception"
},
"status": 400
}
},
"action": [
"create",
"{_index=winlogbeat-default}",
"{agent={name=windows, id=9197309a-3f83-4ec2-bc54-0bb79783b621, type=winlogbeat, ephemeral_id=238e0a1e-ac6f-4451-9ba3-d38f6dd1cdb5, version=8.8.0}, winlog={computer_name=windows.default.example.com, process={pid=4, thread={id=3660}}, keywords=[Audit Failure], channel=Security, event_data={SourceAddress=fe80::ffff:ffff:ffff:ffff, LayerName=%%14610, DestPort=130, RemoteMachineID=S-1-0-0, FilterRTID=95023, SourcePort=0, RemoteUserID=S-1-0-0, Direction=%%14592, LayerRTID=46, ProcessID=4, Protocol=58, DestAddress=ff02::1, Application=System}, opcode=Info, version=1, record_id=184294, task=Filtering Platform Connection, event_id=5157, provider_guid={54849625-5478-4994-a5ba-3e3b0328c30d}, api=wineventlog, provider_name=Microsoft-Windows-Security-Auditing}, log={level=information}, message=The Windows Filtering Platform has blocked a connection.\n\nApplication Information:\n\tProcess ID:\t\t4\n\tApplication Name:\tSystem\n\nNetwork Information:\n\tDirection:\t\tInbound\n\tSource Address:\t\tfe80::ffff:ffff:ffff:ffff\n\tSource Port:\t\t0\n\tDestination Address:\tff02::1\n\tDestination Port:\t\t130\n\tProtocol:\t\t58\n\nFilter Information:\n\tFilter Run-Time ID:\t95023\n\tLayer Name:\t\tReceive/Accept\n\tLayer Run-Time ID:\t46, type=beats, tags=[beats], @timestamp=2024-07-31T09:55:12.012Z, ecs={version=8.0.0}, @version=1, host={hostname=windows, os={build=17763.5576, kernel=10.0.17763.5576 (WinBuild.160101.0800), name=Windows Server 2019 Standard, type=windows, family=windows, version=10.0, platform=windows}, ip=[fe80::168f:f4ec:e986:0000, 10.10.10.10], name=windows, id=42569218-7bbd-4008-83cd-c2fafe8c8a5c, mac=[00-50-FF-FF-FF-FF], architecture=x86_64}, event={code=5157, original=<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-a5ba-3e3b0328c30d}'/><EventID>5157</EventID><Version>1</Version><Level>0</Level><Task>12810</Task><Opcode>0</Opcode><Keywords>0x8010000000000000</Keywords><TimeCreated SystemTime='2024-07-31T09:55:12.012388900Z'/><EventRecordID>184294</EventRecordID><Correlation/><Execution ProcessID='4' ThreadID='3660'/><Channel>Security</Channel><Computer>windows.default.example.com</Computer><Security/></System><EventData><Data Name='ProcessID'>4</Data><Data Name='Application'>System</Data><Data Name='Direction'>%%14592</Data><Data Name='SourceAddress'>fe80::ffff:ffff:ffff:ffff</Data><Data Name='SourcePort'>0</Data><Data Name='DestAddress'>ff02::1</Data><Data Name='DestPort'>130</Data><Data Name='Protocol'>58</Data><Data Name='FilterRTID'>95023</Data><Data Name='LayerName'>%%14610</Data><Data Name='LayerRTID'>46</Data><Data Name='RemoteUserID'>S-1-0-0</Data><Data Name='RemoteMachineID'>S-1-0-0</Data></EventData><RenderingInfo Culture='en-US'><Message>The Windows Filtering Platform has blocked a connection.\r\n\r\nApplication Information:\r\n\tProcess ID:\t\t4\r\n\tApplication Name:\tSystem\r\n\r\nNetwork Information:\r\n\tDirection:\t\tInbound\r\n\tSource Address:\t\tfe80::ffff:ffff:ffff:ffff\r\n\tSource Port:\t\t0\r\n\tDestination Address:\tff02::1\r\n\tDestination Port:\t\t130\r\n\tProtocol:\t\t58\r\n\r\nFilter Information:\r\n\tFilter Run-Time ID:\t95023\r\n\tLayer Name:\t\tReceive/Accept\r\n\tLayer Run-Time ID:\t46</Message><Level>Information</Level><Task>Filtering Platform Connection</Task><Opcode>Info</Opcode><Channel>Security</Channel><Provider>Microsoft Windows security auditing.</Provider><Keywords><Keyword>Audit Failure</Keyword></Keywords></RenderingInfo></Event>, provider=Microsoft-Windows-Security-Auditing, created=2024-08-07T11:23:05.709Z, kind=event, full_event=The Windows Filtering Platform has blocked a connection.\n\nApplication Information:\n\tProcess ID:\t\t4\n\tApplication Name:\tSystem\n\nNetwork Information:\n\tDirection:\t\tInbound\n\tSource Address:\t\tfe80::ffff:ffff:ffff:ffff\n\tSource Port:\t\t0\n\tDestination Address:\tff02::1\n\tDestination Port:\t\t130\n\tProtocol:\t\t58\n\nFilter Information:\n\tFilter Run-Time ID:\t95023\n\tLayer Name:\t\tReceive/Accept\n\tLayer Run-Time ID:\t46, action=Filtering Platform Connection, outcome=failure}}"
],
"status": 400
}
The following processor can turn valid IPv6 address into invalid ones.
https://github.com/elastic/integrations/blob/af36780d5654792f68466c42ddeb95627e464d17/packages/system/data_stream/security/elasticsearch/ingest_pipeline/default.yml#L11-L15
Examples:
fe80::ffff:ffff:fffe
->fe80ffff:fffe
fe80::ffff:ffff:ffff:ffff
->fe80ffff:ffff:ffff
Which then later causes errors such as![image](https://github.com/elastic/integrations/assets/1696711/6f189e04-108c-49ec-9c66-ba004c694d57)
'fe80ffff:ffff:ffff' is not an IP string literal.
in other processors. It also causes other issues.I believe this processor's intent is to transform IPs such as
::ffff:192.168.1.1
into192.168.1.1
. In which case, the pattern should be^::ffff:
to anchor the pattern at the beginning of the string. (See my below comment for a better proposed solution.)I tested this in a simulated pipeline using both
fe80::ffff:ffff:fffe
and::ffff:192.168.1.1
values. It correctly left the former untouched and correctly stripped the prefix from the latter.References: