brexhq / substation

Substation is a toolkit for routing, normalizing, and enriching security event and audit logs.
https://substation.readme.io
MIT License
322 stars 16 forks source link

feat(transform): Add KV Store Set Add Transform #215

Closed jshlbrd closed 2 months ago

jshlbrd commented 2 months ago

Description

Motivation and Context

The backend infra used by the KV store feature (primarily DynamoDB, but in the future Redis or other similar systems) support more operations than setting and getting records, so this is a starting point for adding more support to that part of the project. Stating the obvious, this makes it possible to store items as a set / unique list in a KV store; this adds another capability to track state (as indexed values). It's worth mentioning that sets are unordered lists containing unique items.

This required a rename of the existing KV store functions:

The rename isn't breaking, but references in substation.libsonnet were changed from kv_store.get() and kv_store.set() to kv_store.iget() and kv_store.iset(). This gives us more room to add more KV store functions in the future, like:

How Has This Been Tested?

Types of changes

Checklist: