fkie-cad / Logprep

log data pre processing, generation and shipping in python
https://logprep.readthedocs.io/en/latest/
GNU Lesser General Public License v2.1
30 stars 8 forks source link

OpenSearch Output: Logs in default index don't contain original index #691

Open ppcad opened 3 weeks ago

ppcad commented 3 weeks ago

Logs that are written into the default index using the OpenSearch Output do not contain their original target index, since the original index field must be overwritten to write the logs into the default index. It would be useful for debugging if the original index was written into an additional field (e.g. original_index) analogous to the field reason.

ekneg54 commented 3 weeks ago

As we use opensearch datastreams to ensure the indizis are rotated if they grow upon a specific level we use the default_Index option in another way. For us it is the name of the datastream to write the logs to like the name of a kafka topic. We dont have to set the index during processing.