Open philipsabri opened 4 years ago
Event in the eventviewer XML:
The XML seems broken. I've checked it by some XML validators and all of them report the following error:
The Content Of Elements Must Consist Of Well-formed Character Data Or Markup., Line '22', Column '143'.
It seems that an invalid <
exists in <Data Name="ScriptBlockText">
node:
=== # Configuration #========================================== <# $ConfWindowsUpdateSettings
^ Here!
^ Here!
^ Here!
The XML seems broken.
I agree the XML is broken. Is there something we can do about this or is it up to Microsoft?
Also I didnt know I could save the eventlog as .xml so here is the original .xml file of the event if that would make any difference.
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<Events><Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-PowerShell' Guid='{A0C1853B-5C40-4B15-8766-3CF1C58F985A}'/><EventID>4104</EventID><Version>1</Version><Level>3</Level><Task>2</Task><Opcode>15</Opcode><Keywords>0x0</Keywords><TimeCreated SystemTime='2020-08-05T08:54:21.118916000Z'/><EventRecordID>22338</EventRecordID><Correlation ActivityID='{4FC9EF57-59A7-0001-26CA-FA52A759D601}'/><Execution ProcessID='14424' ThreadID='13956'/><Channel>Microsoft-Windows-PowerShell/Operational</Channel><Computer>x</Computer><Security UserID='x'/></System><EventData><Data Name='MessageNumber'>1</Data><Data Name='MessageTotal'>1</Data><Data Name='ScriptBlockText'>#==========================================
# Configuration
#==========================================
<#
$ConfWindowsUpdateSettings = 'Name;Value
WUServer;http://x:x
WUStatusServer;http://x:x
TargetGroupEnabled;1
DoNotConnectToWindowsUpdateInternetLocations;0
' | ConvertFrom-Csv -Delimiter ";"
#>
$ConfWindowsUpdateSettings = 'Name;Value
WUServer;http://x:x
WUStatusServer;http://x:x
TargetGroupEnabled;1
' | ConvertFrom-Csv -Delimiter ";"
$WarnMissingUpdates = 10
$CritMissingUpdates = 20
$WarnLastInstallDate = -30
$CritLastInstallDate = -60
$Result = ""
$Status = "OK"
$PerfStats = ""
#==========================================
# Function Exit_State
#==========================================
Function Exit_State ($Status)
{
Switch ($Status)
{
"OK" {Return 0}
"WARNING" {Return 1}
"CRITICAL" {Return 2}
"UNKNOWN" {Return 3}
}
}
#==========================================
# Function ExitReport
#==========================================
Function ExitReport ($Status, $Result)
{
Write-Host ($Status + ":" + $Result + "`n")
Exit [int](Exit_State $Status)
}
#==========================================
# Function Max_State
#==========================================
Function Max_State ($Current, $Compare)
{
If ($Compare -eq "CRITICAL" -or $Current -eq "CRITICAL")
{
Return "CRITICAL"
}
ElseIf ($Compare -eq "WARNING" -or $Current -eq "WARNING")
{
Return "WARNING"
}
ElseIf ($Compare -eq "UNKNOWN" -or $Current -eq "UNKNOWN")
{
Return "UNKNOWN"
}
Else
{
Return "OK"
}
}
#Status examples
#$Status = Max_State $Status "OK"
#$Status = Max_State $Status "WARNING"
#$Status = Max_State $Status "CRITICAL"
#$Status = Max_State $Status "UNKNOWN"
#==========================================
# Main
#==========================================
$Error.Clear()
# Verify Windows Update Settings
$TempResult = $null
$WindowsUpdateSettings = Get-ItemProperty "HKLM:\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate"
foreach ($Setting in $ConfWindowsUpdateSettings)
{
If ($WindowsUpdateSettings.($Setting.Name) -ne $Setting.Value)
{
$Status = Max_State $Status "WARNING"
$TempResult += ("{0}," -f $Setting.Name)
}
}
If ($TempResult)
{
$Result += " Settings: {0};" -f $TempResult.TrimEnd(",")
}
Else
{
$Result += " Settings: OK;"
}
# Verify service not disabled
$wuauserv = Get-Service -Name wuauserv
if ($wuauserv.StartType -eq "Disabled") {
Set-Service -Name wuauserv -StartupType Manual
}
#<#
# Get update information from Microsoft update
$UpdateSession = New-Object -ComObject 'Microsoft.Update.Session'
$UpdateSearcher = $UpdateSession.CreateUpdateSearcher()
$SearchResult = $UpdateSearcher.Search("IsInstalled=0 and Type='Software' and IsHidden=0")
$HistoryCount = $UpdateSearcher.GetTotalHistoryCount()
$UpdateHistory = $UpdateSearcher.QueryHistory(0,$HistoryCount)
$MissingUpdates = ($SearchResult.Updates | Where-Object {$_.Title -notlike '*Definition Update*' -and $_.Description -notlike '*Install this update to revise the files that are used to detect viruses, spyware, and other potentially unwanted software.*'} | Measure-Object | Select-Object -ExpandProperty Count)
$LastInstallDate = (Get-Date ($UpdateHistory | Sort-Object -Descending -Property Date | Where-Object {$_.Title -notlike 'Definition Update *' -and $_.Description -notlike '*Install this update to revise the files that are used to detect viruses, spyware, and other potentially unwanted software.*'} | Select-Object -First 1 -ExpandProperty Date) -Format "yyyy-MM-dd HH:mm:ss")
#$LastInstallTitle = ($UpdateHistory | Sort-Object -Descending -Property Date | Where-Object {$_.Title -notlike 'Definition Update *'} | Select-Object -First 1 -ExpandProperty Title)
#>
# Verify missing count
$Result += " Missing: {0};" -f $MissingUpdates
$PerfStats += "Missing={0};{1};{2} " -f $MissingUpdates, $WarnMissingUpdates, $CritMissingUpdates
If ($MissingUpdates -ge $CritMissingUpdates)
{
$Status = Max_State $Status "CRITICAL"
}
ElseIf ($MissingUpdates -ge $WarnMissingUpdates)
{
$Status = Max_State $Status "WARNING"
}
# Verify last install date
$Result += " Last: {0}" -f (Get-Date $LastInstallDate -Format "yyyy-MM-dd")
If ([datetime]$LastInstallDate -lt (Get-Date).AddDays($CritLastInstallDate))
{
$Status = Max_State $Status "CRITICAL"
}
ElseIf ([datetime]$LastInstallDate -lt (Get-Date).AddDays($WarnLastInstallDate))
{
$Status = Max_State $Status "WARNING"
}
# Check for errors
If ($Error.Count -gt 0)
{
$Status = Max_State $Status "UNKNOWN"
$Result += "; Errors: {0}" -f $Error.Count
#$Error | Out-File -FilePath C:\x -Append
}
#==========================================
# Exit and Report
#==========================================
$Result += "|" + $PerfStats
ExitReport $Status $Result
</Data><Data Name='ScriptBlockId'>acad77d5-f5ce-414b-96de-ea6f78ba3021</Data><Data Name='Path'>C:\x.ps1</Data></EventData><RenderingInfo Culture='en-US'><Message>Creating Scriptblock text (1 of 1):
#==========================================
# Configuration
#==========================================
<#
$ConfWindowsUpdateSettings = 'Name;Value
WUServer;http://x:x
WUStatusServer;http://x:x
TargetGroupEnabled;1
DoNotConnectToWindowsUpdateInternetLocations;0
' | ConvertFrom-Csv -Delimiter ";"
#>
$ConfWindowsUpdateSettings = 'Name;Value
WUServer;http://x:x
WUStatusServer;http://x:x
TargetGroupEnabled;1
' | ConvertFrom-Csv -Delimiter ";"
$WarnMissingUpdates = 10
$CritMissingUpdates = 20
$WarnLastInstallDate = -30
$CritLastInstallDate = -60
$Result = ""
$Status = "OK"
$PerfStats = ""
#==========================================
# Function Exit_State
#==========================================
Function Exit_State ($Status)
{
Switch ($Status)
{
"OK" {Return 0}
"WARNING" {Return 1}
"CRITICAL" {Return 2}
"UNKNOWN" {Return 3}
}
}
#==========================================
# Function ExitReport
#==========================================
Function ExitReport ($Status, $Result)
{
Write-Host ($Status + ":" + $Result + "`n")
Exit [int](Exit_State $Status)
}
#==========================================
# Function Max_State
#==========================================
Function Max_State ($Current, $Compare)
{
If ($Compare -eq "CRITICAL" -or $Current -eq "CRITICAL")
{
Return "CRITICAL"
}
ElseIf ($Compare -eq "WARNING" -or $Current -eq "WARNING")
{
Return "WARNING"
}
ElseIf ($Compare -eq "UNKNOWN" -or $Current -eq "UNKNOWN")
{
Return "UNKNOWN"
}
Else
{
Return "OK"
}
}
#Status examples
#$Status = Max_State $Status "OK"
#$Status = Max_State $Status "WARNING"
#$Status = Max_State $Status "CRITICAL"
#$Status = Max_State $Status "UNKNOWN"
#==========================================
# Main
#==========================================
$Error.Clear()
# Verify Windows Update Settings
$TempResult = $null
$WindowsUpdateSettings = Get-ItemProperty "HKLM:\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate"
foreach ($Setting in $ConfWindowsUpdateSettings)
{
If ($WindowsUpdateSettings.($Setting.Name) -ne $Setting.Value)
{
$Status = Max_State $Status "WARNING"
$TempResult += ("{0}," -f $Setting.Name)
}
}
If ($TempResult)
{
$Result += " Settings: {0};" -f $TempResult.TrimEnd(",")
}
Else
{
$Result += " Settings: OK;"
}
# Verify service not disabled
$wuauserv = Get-Service -Name wuauserv
if ($wuauserv.StartType -eq "Disabled") {
Set-Service -Name wuauserv -StartupType Manual
}
#<#
# Get update information from Microsoft update
$UpdateSession = New-Object -ComObject 'Microsoft.Update.Session'
$UpdateSearcher = $UpdateSession.CreateUpdateSearcher()
$SearchResult = $UpdateSearcher.Search("IsInstalled=0 and Type='Software' and IsHidden=0")
$HistoryCount = $UpdateSearcher.GetTotalHistoryCount()
$UpdateHistory = $UpdateSearcher.QueryHistory(0,$HistoryCount)
$MissingUpdates = ($SearchResult.Updates | Where-Object {$_.Title -notlike '*Definition Update*' -and $_.Description -notlike '*Install this update to revise the files that are used to detect viruses, spyware, and other potentially unwanted software.*'} | Measure-Object | Select-Object -ExpandProperty Count)
$LastInstallDate = (Get-Date ($UpdateHistory | Sort-Object -Descending -Property Date | Where-Object {$_.Title -notlike 'Definition Update *' -and $_.Description -notlike '*Install this update to revise the files that are used to detect viruses, spyware, and other potentially unwanted software.*'} | Select-Object -First 1 -ExpandProperty Date) -Format "yyyy-MM-dd HH:mm:ss")
#$LastInstallTitle = ($UpdateHistory | Sort-Object -Descending -Property Date | Where-Object {$_.Title -notlike 'Definition Update *'} | Select-Object -First 1 -ExpandProperty Title)
#>
# Verify missing count
$Result += " Missing: {0};" -f $MissingUpdates
$PerfStats += "Missing={0};{1};{2} " -f $MissingUpdates, $WarnMissingUpdates, $CritMissingUpdates
If ($MissingUpdates -ge $CritMissingUpdates)
{
$Status = Max_State $Status "CRITICAL"
}
ElseIf ($MissingUpdates -ge $WarnMissingUpdates)
{
$Status = Max_State $Status "WARNING"
}
# Verify last install date
$Result += " Last: {0}" -f (Get-Date $LastInstallDate -Format "yyyy-MM-dd")
If ([datetime]$LastInstallDate -lt (Get-Date).AddDays($CritLastInstallDate))
{
$Status = Max_State $Status "CRITICAL"
}
ElseIf ([datetime]$LastInstallDate -lt (Get-Date).AddDays($WarnLastInstallDate))
{
$Status = Max_State $Status "WARNING"
}
# Check for errors
If ($Error.Count -gt 0)
{
$Status = Max_State $Status "UNKNOWN"
$Result += "; Errors: {0}" -f $Error.Count
#$Error | Out-File -FilePath C:\x -Append
}
#==========================================
# Exit and Report
#==========================================
$Result += "|" + $PerfStats
ExitReport $Status $Result
ScriptBlock ID: acad77d5-f5ce-414b-96de-ea6f78ba3021
Path: C:\x.ps1</Message><Level>Warning</Level><Task>Execute a Remote Command</Task><Opcode>On create calls</Opcode><Channel>Microsoft-Windows-PowerShell/Operational</Channel><Provider></Provider><Keywords></Keywords></RenderingInfo></Event></Events>
@ashie I checked the new XML and it has no error
Try setting parse_description false
. That parser seems to assume (more or less) that the format of the "Description" field matches the Security log where you have "Sections" with "Keys" and "Values" that are tab separated. The relevant code for parsing Description is at https://github.com/fluent/fluent-plugin-windows-eventlog/blob/e061144738ddd3c54971a6966d73a98335ae473f/lib/fluent/plugin/in_windows_eventlog2.rb#L357-L401.
That said, you will need something else to help pull data out of the Description field to help make it relevant.
We noticed our Elastic couldnt parse some logs coming from the Powershell channel, and after some investigation and getting the clean json log directly from
windows_eventlog2
and file output we could clearly see its the parser from this plugin that makes them escape the json value, or not putting them inside the right key at all.Example log:
We can cleary see that
"elseif_($compare_-eq_\"unknown\"_-or_$current_-eq_\"unknown\")\r\n____"
is a json key when it looks like it should be in theDescriptionTitle
, or might even have its own key namedScriptBlockText
after watching th eventviewer xml.Event in the eventviewer XML:
Also as we see here, there is a lot of information missing in the
ScriptBlockText
when fluentd picking it upfluentd config: