search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
29 stars 446 forks source link

[AWS] Add parse_aws_vpc_flow_log processor and format variable into vpcflow logs #6080

Open kaiyan-sheng opened 1 year ago

kaiyan-sheng commented 1 year ago

format variable is added into Filebeat to enable users to specify a custom VPC flow log format. We should also add this feature into the integration.

elasticmachine commented 1 year ago

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

andrewkroh commented 1 year ago

Background: I did not add this feature to integrations because it would require shifting processing toward the edge, and to date all integrations have been moving toward centralized processing with Elasticsearch Ingest Node. So this would have been counter to that goal of having the transformations done in ES.

There are no technical limitations to using the Filebeat processor to do the parsing. Or alternatively the format feature could be implemented through Ingest Node.