[Request] SMTP/Extended HTTP Props

[KillerInstinct]

Any chance you'd be able to add in props for SMTP fields and extended HTTP logging? (Headers mostly)

[anthonygtellez]

If you could provide a sanitized sample event in eve.json format I'd be more than happy to update the props/transforms. I don't have a mail server configured in my test lab at the moment to generate some logs. :)

[KillerInstinct]

Here's a few events for you with notes just above them: https://gist.github.com/KillerInstinct/3ba2a62927cda666e2ff05e29c4f7c71

Rather annoying to gather this as Suri only logs what it needs to. I don't have any examples of a log with with email.bcc{}="*" (existing really) but It's safe to assume that would be a multivalue field just like email.to/email.cc (in the notes as well)

Full list of fields (from Suricata.yaml): https://github.com/inliniac/suricata/blob/master/suricata.yaml.in#L214

It would seem I also don't have any logs where the 'relays' and 'x-originating-ip' are populated, so I'm unsure what the props would look like for that. I can probably ask around for some pcaps if you want to make the Suri SMTP -> CIM more complete.

[anthonygtellez]

Thanks, sorry for the delay, I haven't had a chance to update the code. I'll have something for you in a few days. :)

[KillerInstinct]

No worries. I presume for your alert/ids aliases you have the additional logging context turned off (these)? EG in your alert logs you don't have http/tls data like such:

{
    "timestamp": "2016-12-13T13:57:14.757149-0500",
    "flow_id": 1917482856792595,
    "in_iface": "p1p1",
    "event_type": "alert",
    "src_ip": "52.203.95.194",
    "src_port": 443,
    "dest_ip": "[redacted]",
    "dest_port": 62767,
    "proto": "TCP",
    "tx_id": 0,
    "alert": {
        "action": "allowed",
        "gid": 1,
        "signature_id": 2230003,
        "rev": 1,
        "signature": "SURICATA TLS invalid handshake message",
        "category": "Generic Protocol Command Decode",
        "severity": 3
    },
    "tls": {
        "subject": "C=US, ST=California, L=San Jose, O=Adobe Systems Incorporated, OU=Digital Marketing, CN=*.demdex.net",
        "issuerdn": "C=US, O=DigiCert Inc, OU=www.digicert.com, CN=DigiCert SHA2 High Assurance Server CA",
        "fingerprint": "bd:e4:6e:a7:61:37:33:45:e7:85:7e:2b:4d:9b:42:c9:f7:af:96:f1",
        "sni": "dpm.demdex.net",
        "version": "TLS 1.2"
    }
}

There's a couple of other fields for turning HTTP data on as well, which is useful for the IDS alerts in applying context to the analyst. Just lemme know if you want examples.

-KI

[anthonygtellez]

Thanks for this additional event. Check out commit 98534777102c14ec9ae17fb842ec69bcc14226b5 for the SMTP props. Let me know if I'm still missing something, the example log didn't have every field that exists for CIM. I may need to create some evals for those depending on what's available in the normal & extended Suricata output.

[KillerInstinct]

Looks good, thanks!