Inconsistent data type parsing between JSON and XML

[ancailliau]

JSON parser parses 5 as an integer while XML parser parses as a string. We can't write a rule that would match on the same events represented in XML or JSON.

Assuming two simple examples test.json

{
  "Event": {
    "System": {
      "Provider": "Linux-Sysmon",
      "TimeCreated": "2022-08-24T08:35:37.343436000Z",
      "EventID": 5
    }
  }
}

and test.xml

<?xml version="1.0"?>
<Events>
  <Event>
    <System>
      <Provider>Linux-Sysmon</Provider>
      <TimeCreated>2022-08-24T08:35:37.343436000Z</TimeCreated>
      <EventID>5</EventID>
    </System>
  </Event>
</Events>

With their respective mappings:

---
name: Test Sigma mappings for JSON
kind: json
rules: sigma

groups:
  - name: Sigma
    timestamp: TimeCreated
    filter:
      Provider: "*"
    fields:
      - name: Event ID
        from: EventID
        to: Event.System.EventID
      - name: Provider
        from: Provider
        to: Event.System.Provider
      - name: TimeCreated
        from: TimeCreated
        to: Event.System.TimeCreated

---
name: Test Sigma mappings for XML
kind: xml
rules: sigma

groups:
  - name: Sigma
    timestamp: TimeCreated
    filter:
      Provider: "*"
    fields:
      - name: Event ID
        from: EventID
        to: Event.System.EventID.$value
      - name: Provider
        from: Provider
        to: Event.System.Provider.$value
      - name: TimeCreated
        from: TimeCreated
        to: Event.System.TimeCreated.$value

This Sigma rule only triggers on the JSON file.

title: Test rule
id: a33c1924-82f0-4ec5-a5f7-53954dbf9638
status: experimental
description: Test rule
author: Antoine Cailliau
date: 2022/08/24
detection:
    selection:
        EventID: 5
    condition: selection
level: low
falsepositives:
    - Unknown

This Sigma rule only triggers on the XML file.

title: Test rule
id: a33c1924-82f0-4ec5-a5f7-53954dbf9638
status: experimental
description: Test rule
author: Antoine Cailliau
date: 2022/08/24
detection:
    selection:
        EventID: '5'
    condition: selection
level: low
falsepositives:
    - Unknown

A single rule is expected to match on both JSON and XML file.

[alexkornitzer]

Apologies for the late reply, was going to respond this morning but I got waylaid at the office!

Yeah so this is a pretty standard problem when it comes to generalised rule formats, I thought I had added type coercing support into the mapping files but I must have forgotten. I will get that done tomorrow, it will probably end up looking something like this:

...
      - name: Event ID
        from: EventID
        to: int(Event.System.EventID.$value)
...

Thanks for the thorough minimal example btw, that will make it much easier for me to add the feature in.

[alexkornitzer]

@ancailliau when you get time if you could test the master branch that would be great. Thanks again for filing this as I found a bug in the mapper as well :)

[ancailliau]

Sorry, I did not see your comment. I'll test as soon as possible and provides feedback.