KristianLyng
commented
2 years ago I've added some updates:
- The enricher transformer no longer has a source field or any local method of updating itself
- The enricherUpdater sender is added which updates the enricher transformer (ok, slightly confusing, but not THAT bad)
- Examples are updated
- The WholeData receiver can be used to replace the Source-field. The example provided demonstrates this.
One annoyance that's left is the need for timestamps that are never used.
I've added an experimental enrichment transformer, which allows an operator to use skogul to add fields to metrics based on metadata. E.g.: Add description to interface names from a database.
This is the simplest possible implementation I could make, and will definitely change as the idea matures. As such, the documentation is somewhat deliberately spotty. Instead, use this issue until things mature.
It currently works by parsing a JSON document consisting of an array of metrics, though timestamp is ignored. See docs/examples/payload/enrich.json for an example, and docs/examples/tester_to_stdout_enrich.json for config example.
Each metric will require metadata, which is what will be matched, and data, which is what will be added to the metadata struct. I appreciate that this is somewhat non-intuitive, and it was mainly done for convenience, and I am guessing it wont actually be a Metric data structure in the future.
Additionally, only metadatafields listed in the "keys" is actually used, which is an obvious short coming, but necessary.
I see numerous places where this needs to be improved, but I wanted to get something out fast, to test with production data/traffic before I start optimizing and improving things.
Please use this thread for general feedback. For obvious bugs, separate issues are fine too of course.