[Enhancement] Update the mapping for the `winlog.event_data.AttributeValue` field

[w0rk3r]

The problem

We need to cover some security descriptor changes in Active Directory events (and other use cases that use this field), and with the current dynamic parsing, we cannot search this field as it often contains more than 1024 chars, and if we could, the keyword field type would make us unable to do partial matches efficiently (they need to be case-insensitive).

[jamiehynds]

@andrewkroh WDYT about increasing or removing the ignore_above for this field to pave the way for some detection rules based on this field? @w0rk3r based on what you've seen, is there a limit you'd recommend for ignore_above or is it too difficult to say?

[w0rk3r]

We may also require to change the type, similar to what we have done in https://github.com/elastic/integrations/issues/1776

[andrewkroh]

Would a match_only_text data type be best for the types of searches you want to do? Perhaps we could create a multi-field where winlog.event_data.AttributeValue.text is match_only_text.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[w0rk3r]

Would a match_only_text data type be best for the types of searches you want to do? Perhaps we could create a multi-field where winlog.event_data.AttributeValue.text is match_only_text.

@andrewkroh match_only_text data type doesn't support analyzers? I think we will have trouble searching through the security descriptors as -, ;;, () are very relevant here and they are ignored by the standard analyzer as far as I remember. I like the idea of having a new field (multi) to not break anything

[andrewkroh]

You're right. So I think we need to work on the configuration of an analyzer that can split the SDDL string. Here's an analyzer I was playing with that might work for this. We might need to create a separate field that specifically holds security descriptors because IIUC the AttributeValue field could hold other types of values that are not security descriptors and hence the SDDL custom analyzer wouldn't be applicable?

``` DELETE my-index-000001 PUT my-index-000001 { "settings": { "analysis": { "analyzer": { "sddl_analyzer": { "tokenizer": "sddl_tokenizer", "filter": [ "uppercase" ] } }, "tokenizer": { "sddl_tokenizer": { "type": "pattern", "pattern": "[OGDS]:|Exists|Any_of|Member_of|Contains|==|!=|<=|>=|[><]|[ ,;\")(!}{]|[|&]{2}" } } } }, "mappings": { "properties": { "sddl": { "type": "keyword", "fields": { "text": { "type": "text", "analyzer": "sddl_analyzer", "search_analyzer": "sddl_analyzer" } } } } } } POST my-index-000001/_analyze { "analyzer": "sddl_analyzer", "text": "O:DAG:DAD:(A;;RPWPCCDCLCR CWOWDSDSW;;;SY)(A;;RPWPCCDCLCRCWOWDSDSW;;;DA)(OA;;CCDC;bf967aba-0de6-11d0-a285-00aa003049e2;;AO)(OA;;CCDC;bf967a9c-0de6-11d0-a285-00aa003049e2;;AO)(OA;;CCDC;6da8a4ff-0e52-11d0-a286-00aa003049e2;;AO)(OA;;CCDC;bf967aa8-0de6-11d0-a285-00aa003049e2;;PO)(A;;RPLCRC;;;AU)S:(AU;SAFA;WDWOSDWPCCDCSW;;;WD)" } POST my-index-000001/_analyze { "analyzer": "sddl_analyzer", "text": "O:DAG:DAD:(XA; ;FX;;;S-1-1-0; (@User.Title==\"PM\" && (@User.Division== \"Finance\" || @User.Division == \"Sales\")))" } POST my-index-000001/_analyze { "analyzer": "sddl_analyzer", "text": "D:(A;;GA;;;SY)(A;;RC;;;OW)(A;;GA;;;S-1-5-80-123231216-2592883651-3715271367-3753151631-4175906628)" } POST my-index-000001/_analyze { "analyzer": "sddl_analyzer", "text": "S-1-5-80-123231216-2592883651-3715271367-3753151631-4175906628" } POST my-index-000001/_analyze { "analyzer": "sddl_analyzer", "text": "D:(XA; ;FX;;;S-1-1-0; (@User.Project Any_of @Resource.Project))" } POST my-index-000001/_analyze { "analyzer": "sddl_analyzer", "text": "D:(XA; ;FR;;;S-1-1-0; (Member_of {SID(Smartcard_SID), SID(BO)} && @Device.Bitlocker))" } POST my-index-000001/_bulk { "create" : {} } { "sddl" : "D:(A;;GA;;;SY)(A;;RC;;;OW)(A;;GA;;;S-1-5-80-123231216-2592883651-3715271367-3753151631-4175906628)" } { "create" : {} } { "sddl" : "O:DAG:DAD:(A;;RPWPCCDCLCR CWOWDSDSW;;;SY)(A;;RPWPCCDCLCRCWOWDSDSW;;;DA)(OA;;CCDC;bf967aba-0de6-11d0-a285-00aa003049e2;;AO)(OA;;CCDC;bf967a9c-0de6-11d0-a285-00aa003049e2;;AO)(OA;;CCDC;6da8a4ff-0e52-11d0-a286-00aa003049e2;;AO)(OA;;CCDC;bf967aa8-0de6-11d0-a285-00aa003049e2;;PO)(A;;RPLCRC;;;AU)S:(AU;SAFA;WDWOSDWPCCDCSW;;;WD)" } { "create" : {} } { "sddl" : "O:DAG:DAD:(XA; ;FX;;;S-1-1-0; (@User.Title==\"PM\" && (@User.Division==\"Finance\" || @User.Division ==\"Sales\")))" } { "create" : {} } { "sddl" : "D:(XA; ;FX;;;S-1-1-0; (@User.Project Any_of @Resource.Project))" } GET my-index-000001/_search { "query": { "match": { "sddl.text": { "query": "BF967a9c-0de6-11d0-a285-00aa003049e2" } } } } ```

[w0rk3r]

IIUC the AttributeValue field could hold other types of values that are not security descriptors and hence the SDDL custom analyzer wouldn't be applicable?

Yup, I mean an analyzer similar to what we've done to PowerShell stuff in https://github.com/elastic/integrations/pull/1931/files to not split on these chars so we can search for SIDs for example, or specific strings that contain those characters

[w0rk3r]

@andrewkroh @jamiehynds just a ping around this, as it is blocking some use cases we want to develop😅

[jamiehynds]

@nfritts is this something your team could look into? It's currently blocking some important detection rules the TRADE team want to develop.