Amazon Security Lake integration - Save intermediate JSON files

wazuh / wazuh-indexer

Wazuh indexer, the Wazuh search engine

https://opensearch.org/docs/latest/opensearch/index/

Apache License 2.0

11 stars 17 forks source link

Amazon Security Lake integration - Save intermediate JSON files #216

Closed AlexRuiz7 closed 5 months ago

AlexRuiz7 commented 5 months ago

Description

Related issue: #128

The integration to Amazon Security Lake first maps the raw Wazuh Events to OCSF, and then encodes the data as Parquet. The Wazuh Events mapped to OCSF as JSON format are not exported. We think it could be interesting to save them as intermediate JSON files. That could be also useful for debugging (see https://github.com/aws-samples/amazon-security-lake-ocsf-validation).

Tasks

[x] Extend the integration to save OCSF mapped events as JSON to an S3 bucket.

AlexRuiz7 commented 5 months ago

S3_BUCKET_OCSF environment variable will define the S3 bucket to write OCSF data to.