search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
23 stars 435 forks source link

aws.securityhub_findings: Update datastream to leverage Cloud Security workflows #11038

Open kcreddy opened 1 month ago

kcreddy commented 1 month ago

As part of supporting AWS SecurityHub for native Cloud Security (CSPM, KSPM, & CNVM) workflows, the following changes are required:

More details in the meta issue https://github.com/elastic/security-team/issues/9961

elasticmachine commented 1 month ago

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

kcreddy commented 1 month ago

PR for mapping changes: https://github.com/elastic/integrations/pull/11158. Will be ready for review after mapping clarifications answered and implemented by Cloud Security team.

kcreddy commented 1 month ago

All clarifications are provided by Cloud Team except for ones still pending for Wiz (1:many mappings for benchmark fields). There is one clarification/suggestion that stands out that may impact the workflow, hence mentioning here:

AWS SecurityHub Finding can contain multiple resources, which provides all the resources effected by that finding. But this multiple resources case wasn't observed in any of the ingested data in the Demo cluster so far. Also, the native CSPM workflow expects i.e., a 1:1 mapping for finding:resource. So, we are taking this approach:

If there is one resource we do what we have in wiz and native - just populate resource.* as keywords, so that if three is one it is a string and if many then array. Arrays won’t work in general for the current ui, but as we don’t see them we can take this trade off for now hoping that it’s a rare case and not many findings will be affected.

cc: @maxcold

kcreddy commented 4 weeks ago

After recommendation from Cloud Team, both the mapping changes: https://github.com/elastic/integrations/issues/11040 and transform changes: https://github.com/elastic/integrations/issues/11039 are now part of same PR: https://github.com/elastic/integrations/pull/11158 This is to make the e2e testing for AWS SecurityHub with CSPM easier and prevent bug fixes in multiple stages. cc: @maxcold, @narph