SigmaHQ / pySigma-backend-elasticsearch

pySigma Elasticsearch backend
GNU Lesser General Public License v3.0
42 stars 26 forks source link

No support for multi-field mappings #22

Closed cospirho closed 1 year ago

cospirho commented 1 year ago

Some fields have multiple mappings. For example, using a text mapping with .text at the end. In the current winlogbeats config, process.executable is a keyword, but process.executable.text is a text field (case insensitive), so we should query on process.executable.text (or make process.executable case insensitive, but this way would result in cleaner, and presumably more performant, queries).

The legacy converter had the keyword_blacklist backend option, but while it usually worked for most deployments, it really wasn't ideal. You can have any mapping with any name you want, so ideally you would specify which fields map to which extensions. Then you could query on process.executable.anything

Thank you for this cool project ^_^

andurin commented 1 year ago

Hi @cospirho,

I'm not sure if I understand you correctly, so please correct me if I am on the wrong way.

The processing pipelines are responsible to do the field mappings like ecs_windows

That would be the easiest point to transform field mappings to your needs.

What I am unsure about as time of writing: Are there significant changes to the queries based on the ES mapping type?

cospirho commented 1 year ago

Oh if that's the intended way, yeah, that would work just fine. We were converting from the old converter, and we were using the options mentioned to do it (for text/keyword fields), noticing they weren't present in this version. Seems they also weren't necessary in the old converter either.

Thanks for clarifying!