search

omerbenamram / evtx

A Fast (and safe) parser for the Windows XML Event Log (EVTX) format
Apache License 2.0
663 stars 64 forks source link

Invalid behaviour when parsing Evtx from Windows Event Forwarding #201

Closed Crypt0-M3lon closed 2 years ago

Crypt0-M3lon commented 2 years ago

Hi,

We are trying to use your library to parse Windows logs but we encountered some strange error when parsing EVTX files coming from a Windows Event Collector server. In the event viewer, the XML is the following:

- <EventData>
  <Data>Set-Mailbox</Data> 
  <Data>-Identity "Administrateur" -DeliverToMailboxAndForward "False" -ForwardingSmtpAddress "smtp:test2@example.com"</Data> 
  <Data>ave.local/Users/Administrateur</Data> 
  <Data>S-1-5-21-186559946-3925841745-111227986-500</Data> 
  <Data>S-1-5-21-186559946-3925841745-111227986-500</Data> 
  <Data>Remote-ManagementShell-Unknown</Data> 
  <Data>5668 w3wp#MSExchangePowerShellAppPool</Data> 
  <Data /> 
  <Data>5</Data> 
  <Data>00:00:26.0389557</Data> 
  <Data>Afficher la forêt entière : 'False', Portée par défaut : « ave.local », Configuration du contrôleur de domaine : « DC.ave.local », Catalogue global préféré : « DC.ave.local », Contrôleurs de domaine préférés : « { DC.ave.local } »</Data> 
  <Data /> 
  <Data /> 
  <Data /> 
  <Data /> 
  <Data /> 
  <Data /> 
  <Data>False</Data> 
  <Data /> 
  <Data>0 objects execution has been proxied to remote server.</Data> 
  <Data /> 
  <Data /> 
  <Data>0</Data> 
  <Data>ActivityId: a3591746-a27b-447a-b8be-ff54ae3a46f1</Data> 
  <Data>ServicePlan:;IsAdmin:True;</Data> 
  <Data /> 
  <Data>fr-FR</Data> 
  </EventData>

If we convert the original EVTX, we obtain the following JSON:

      "Data": {
        "#text": [
          "Set-Mailbox",
          "-Identity \"Administrateur\" -DeliverToMailboxAndForward \"False\" -ForwardingSmtpAddress \"smtp:test2@example.com\"",
          "ave.local/Users/Administrateur",
          "S-1-5-21-186559946-3925841745-111227986-500",
          "S-1-5-21-186559946-3925841745-111227986-500",
          "Remote-ManagementShell-Unknown",
          "5668 w3wp#MSExchangePowerShellAppPool",
          "",
          "5",
          "00:00:26.0389557",
          "Afficher la forêt entière : 'False', Portée par défaut : « ave.local », Configuration du contrôleur de domaine : « DC.ave.local », Catalogue global préféré : « DC.ave.local », Contrôleurs de domaine préférés : « { DC.ave.local } »",
          "",
          "",
          "",
          "",
          "",
          "",
          "False",
          "",
          "0 objects execution has been proxied to remote server.",
          "",
          "",
          "0",
          "ActivityId: a3591746-a27b-447a-b8be-ff54ae3a46f1",
          "ServicePlan:;IsAdmin:True;",
          "",
          "fr-FR"
        ]

[MSExchange_Management.zip](https://github.com/omerbenamram/evtx/files/7571802/MSExchange_Management.zip)
}
    },

But when the log has been forwarding using WEF, when the EVTX is parsed, we obtain the following JSON:

    "EventData": {
      "Data": {
        "#text": "fr-FR"
      }
    },

As you can see, almost all the information are lost. If you want to make some tests, the EVTX are here: MSExchange_Management.zip

Thank you!

omerbenamram commented 2 years ago

Hi, thanks for letting me know.

This is because EventData Data nodes usually either contain a Name key, or appear by their own. For some reason when the event was passed through WEF in your case, it split the Data node into many smaller ones.

In XML it's legal to have many nodes with the same name, but this is illegal in JSON so the parser outputs them with a postfix. Changing the parser to treat Data in this case like any other node produces an output like this:

{
  "Event": {
    "#attributes": {
      "xmlns": "http://schemas.microsoft.com/win/2004/08/events/event"
    },
    "EventData": {
      "Data": "fr-FR",
      "Data_1": "Set-Mailbox",
      "Data_10": "Afficher la forêt entière : 'False', Portée par défaut : « ave.local », Configuration du contrôleur de domaine : « DC.ave.local », Catalogue global préféré : « DC.ave.local », Contrôleurs de domaine préférés : « { DC.ave.local } »",
      "Data_11": "False",
      "Data_12": "0 objects execution has been proxied to remote server.",
      "Data_13": "0",
      "Data_14": "ActivityId: a3591746-a27b-447a-b8be-ff54ae3a46f1",
      "Data_15": "ServicePlan:;IsAdmin:True;",
      "Data_2": "-Identity \"Administrateur\" -DeliverToMailboxAndForward \"False\" -ForwardingSmtpAddress \"smtp:test2@example.com\"",
      "Data_3": "ave.local/Users/Administrateur",
      "Data_4": "S-1-5-21-186559946-3925841745-111227986-500",
      "Data_5": "S-1-5-21-186559946-3925841745-111227986-500",
      "Data_6": "Remote-ManagementShell-Unknown",
      "Data_7": "5668 w3wp#MSExchangePowerShellAppPool",
      "Data_8": "5",
      "Data_9": "00:00:26.0389557"
    },
    "System": {
      "Channel": "MSExchange Management",
      "Computer": "WEC.ave.local",
      "EventID": {
        "#attributes": {
          "Qualifiers": "16384"
        },
        "#text": "1"
      },
      "EventRecordID": "3229",
      "Keywords": "0x80000000000000",
      "Level": "4",
      "Provider": {
        "#attributes": {
          "Name": "MSExchange CmdletLogs"
        }
      },
      "Security": null,
      "Task": "1",
      "TimeCreated": {
        "#attributes": {
          "SystemTime": "2021-11-19T16:52:33.833733500Z"
        }
      }
    }
  }
}

I think this is a fair representation of the XML, what do you think?

Crypt0-M3lon commented 2 years ago

Thanks you for the very clear technical explanation. I think it would be a better solution to have the same output for the same log (from WEF or not) even if the binxml representation is slightly different, in the event viewer they basically look the same. Having different behavior is making logs exploitation difficult.

omerbenamram commented 2 years ago

The event log viewer simply goes and prints everything that is Data to the log output. So having

<EventData>
    <Data> A String </Data>
</EventData>

and 

<EventData>
    <Data> A </Data>
    <Data> String </Data>
</EventData>

Looks the same textually - but I'm not sure whether can be parsed the same.

Manipulating the JSON output is OK by me - since it's an interpretation on the xml structure and has no one-to-one representation, but I don't think we can do anything about the XML.

I could have Data be concatenated when the Name is empty when outputting to JSON, I'll see if its simple enough.

omerbenamram commented 2 years ago

It would look like this:

{
  "Event": {
    "#attributes": {
      "xmlns": "http://schemas.microsoft.com/win/2004/08/events/event"
    },
    "EventData": {
      "Data": {
        "#text": "Set-Mailbox-Identity \"Administrateur\" -DeliverToMailboxAndForward \"False\" -ForwardingSmtpAddress \"smtp:test2@example.com\"ave.local/Users/AdministrateurS-1-5-21-186559946-3925841745-111227986-500S-1-5-21-186559946-3925841745-111227986-500Remote-ManagementShell-Unknown5668 w3wp#MSExchangePowerShellAppPool500:00:26.0389557Afficher la forêt entière : 'False', Portée par défaut : « ave.local », Configuration du contrôleur de domaine : « DC.ave.local », Catalogue global préféré : « DC.ave.local », Contrôleurs de domaine préférés : « { DC.ave.local } »False0 objects execution has been proxied to remote server.0ActivityId: a3591746-a27b-447a-b8be-ff54ae3a46f1ServicePlan:;IsAdmin:True;fr-FR"
      }
    },
    "System": {
      "Channel": "MSExchange Management",
      "Computer": "WEC.ave.local",
      "EventID": {
        "#attributes": {
          "Qualifiers": "16384"
        },
        "#text": "1"
      },
      "EventRecordID": "3229",
      "Keywords": "0x80000000000000",
      "Level": "4",
      "Provider": {
        "#attributes": {
          "Name": "MSExchange CmdletLogs"
        }
      },
      "Security": null,
      "Task": "1",
      "TimeCreated": {
        "#attributes": {
          "SystemTime": "2021-11-19T16:52:33.833733500Z"
        }
      }
    }
  }
}
Crypt0-M3lon commented 2 years ago

Why not the following json that is the output for the non-wec EVTX file? I think it's more representative of the orginal XML

{
  "Event": {
    "#attributes": {
      "xmlns": "http://schemas.microsoft.com/win/2004/08/events/event"
    },
    "EventData": {
      "Binary": null,
      "Data": {
        "#text": [
          "Set-Mailbox",
          "-Identity \"Administrateur\" -DeliverToMailboxAndForward \"False\" -ForwardingSmtpAddress \"smtp:test2@example.com\"",
          "ave.local/Users/Administrateur",
          "S-1-5-21-186559946-3925841745-111227986-500",
          "S-1-5-21-186559946-3925841745-111227986-500",
          "Remote-ManagementShell-Unknown",
          "5668 w3wp#MSExchangePowerShellAppPool",
          "",
          "5",
          "00:00:26.0389557",
          "Afficher la forêt entière : 'False', Portée par défaut : « ave.local », Configuration du contrôleur de domaine : « DC.ave.local », Catalogue global préféré : « DC.ave.local », Contrôleurs de domaine préférés : « { DC.ave.local } »",
          "",
          "",
          "",
          "",
          "",
          "",
          "False",
          "",
          "0 objects execution has been proxied to remote server.",
          "",
          "",
          "0",
          "ActivityId: a3591746-a27b-447a-b8be-ff54ae3a46f1",
          "ServicePlan:;IsAdmin:True;",
          "",
          "fr-FR"
        ]
      }
    },
    "System": {
      "Channel": "MSExchange Management",
      "Computer": "WEC.ave.local",
      "EventID": {
        "#attributes": {
          "Qualifiers": 16384
        },
        "#text": 1
      },
      "EventRecordID": 3229,
      "Keywords": "0x80000000000000",
      "Level": 4,
      "Provider": {
        "#attributes": {
          "Name": "MSExchange CmdletLogs"
        }
      },
      "Security": null,
      "Task": 1,
      "TimeCreated": {
        "#attributes": {
          "SystemTime": "2021-11-19T16:52:33.833733Z"
        }
      }
    }
  }
}
forensicmatt commented 2 years ago

Why not the following json that is the output for the non-wec EVTX file? I think it's more representative of the orginal XML

{
  "Event": {
    "#attributes": {
      "xmlns": "http://schemas.microsoft.com/win/2004/08/events/event"
    },
    "EventData": {
      "Binary": null,
      "Data": {
        "#text": [
          "Set-Mailbox",
          "-Identity \"Administrateur\" -DeliverToMailboxAndForward \"False\" -ForwardingSmtpAddress \"smtp:test2@example.com\"",
          "ave.local/Users/Administrateur",
          "S-1-5-21-186559946-3925841745-111227986-500",
          "S-1-5-21-186559946-3925841745-111227986-500",
          "Remote-ManagementShell-Unknown",
          "5668 w3wp#MSExchangePowerShellAppPool",
          "",
          "5",
          "00:00:26.0389557",
          "Afficher la forêt entière : 'False', Portée par défaut : « ave.local », Configuration du contrôleur de domaine : « DC.ave.local », Catalogue global préféré : « DC.ave.local », Contrôleurs de domaine préférés : « { DC.ave.local } »",
          "",
          "",
          "",
          "",
          "",
          "",
          "False",
          "",
          "0 objects execution has been proxied to remote server.",
          "",
          "",
          "0",
          "ActivityId: a3591746-a27b-447a-b8be-ff54ae3a46f1",
          "ServicePlan:;IsAdmin:True;",
          "",
          "fr-FR"
        ]
      }
    },
    "System": {
      "Channel": "MSExchange Management",
      "Computer": "WEC.ave.local",
      "EventID": {
        "#attributes": {
          "Qualifiers": 16384
        },
        "#text": 1
      },
      "EventRecordID": 3229,
      "Keywords": "0x80000000000000",
      "Level": 4,
      "Provider": {
        "#attributes": {
          "Name": "MSExchange CmdletLogs"
        }
      },
      "Security": null,
      "Task": 1,
      "TimeCreated": {
        "#attributes": {
          "SystemTime": "2021-11-19T16:52:33.833733Z"
        }
      }
    }
  }
}

I like the array approach better than a catted string. I will see if I can make a PR to implement this.

Crypt0-M3lon commented 2 years ago

Hello, any news about this issue?

Crypt0-M3lon commented 2 years ago

Wo, I was about to make a PR with the same kind of implementation but probably less proper, thank you! :D Will you publish a new version of the tool (and one of the python binding)?