search

aws-samples / siem-on-amazon-opensearch-service

A solution for collecting, correlating and visualizing multiple types of logs to help investigate security incidents.
MIT No Attribution
570 stars 187 forks source link

Integration of AWS-Organization #264

Open AlexanderMannsfeld0007 opened 2 years ago

AlexanderMannsfeld0007 commented 2 years ago

Hi,

at first I am totally in love with this, and my colleagues also!

My suggestion is the inclusion of following architecture, a bit analogue to the geoIp:

  1. A new Bucket, "SIEM-Orga-data"
  2. A new lambda exec-role on the management account
  3. A new lambda running on the siem-account, fetching data from the Management-account via 2. and dumping it into the bucket in 1., nicely packaged as json:
    • GreatparenOu,ParentOu, AccountId
    • for every OU and accountId the tagging (key and value)
  4. extending the aes-es-siem-loader to take use of the data (eg, via accountid adding the information to the event).

This would help greatly in creating orga-specific reports and dashboards, and by the integration of the tagging it becomes even more dynamic. We for example have tags like contact adresses for technical responsibles, contacts in general, webhooks for alerting etc.

Just imagine having reports based on events with "access_denied", grouped by "contact adress" :-).

Thanks in advance & Cheers,

Alexander

nakajiak commented 2 years ago

Thanks for the feedback. Your request is to enrith logs with some rules or logic, right? We can use these ECS fields

AlexanderMannsfeld0007 commented 2 years ago

Hi, exactly!

Regular, either the parent OrganizationUnit or tagging on the accounts itself is used for a lot of aggregation / clusterization (and in our management-accounts also for a lot of reporting already).

Additional, to the cases I have mentioned about regarding reporting per team / context, we additional can use this for an elevated permission-management:

Eg, we integrate in all events the "aws organization parent OU" of the accounts, we then can create roles with document based permission policies. These roles then are assigne an IAM backend-role, also reflection the OU-name (or a tag-value). In our Cognito IDP-pools (in our case based on SAMl via AzureAD), we then would assign the IAM-role as the auth-role via a rule based on the Claim.

By this an organisation can build a huge oopensearch-domain for an entire-organisation, offering views for the governance & security teams and also having roles for the teams, automatically filtering only the events belonging to them.

Cheers,

Alexander

AlexanderMannsfeld0007 commented 2 years ago

Hi, meanwhile we came a bit further - unfortunately not a state I would propose as a pull-request .. but: We added another bucket for the orga-metadata (actually we dumped our whole aws organization as a json, the whole tree, from root, OUs, the OU tagging down to the accounts and their tagging). Then as point of entry we used the enrich-class to add our fields to every event, based on the account-id, like "labels.account_email labels.account_name labels.environment labels.ou_id" .. etc pp.

Worked well. log-aws-cloudtrail has now our orga-data, and after adding another custom mapping template also are searchable.

Didnt worked so well with log-aws- and log-.

So we extended the mapping (for every index, with every field of the custom payload set to index:true) .. still our customfields are not searchable in the meta-indexes (log-aws- and log-). Today I have expanded (also a bit dirty) the "transform_to_ecs"-function (and also added mappings to the aws.ini).. Checked the result via logger and it appears like the tagging is written sucessfully into the ecs-object (as "labels").

I reindexed the indexes in hot- and deleted everything from ultra-warm & cold-storage .. still the new fields are not searchable from log-aws- and log- (important to create opensearch-roles with field-permissions, to map them to team-specific-backend-(IAM)-roles, connected to cognito user-pools ..).

I also refreshed the index mappings.

Do you have, please, any hint where also to look?

Thanks in advance & Cheers,

Alexander

Ps: A lot of dirtyness resulted out of when "cloud.account.id" is filled. Right now I am running "get_value_and_input_into_ecs_dict" twice, the first time as supposed, the second time as I need "cloud.account.id" for the "orga-lookup" set properly.

AlexanderMannsfeld0007 commented 2 years ago

Minor update, I discovered that "index.max_docvalue_fields_search" was set a bit to low and therefor limited to much the indexed fields in the index-patterns log- log-aws- .. I set it a bit higher and temporarily the fields appeared.

AlexanderMannsfeld0007 commented 2 years ago

Hi, finally everything has come together. Every event is enriched with some meta-data from our AWS organizations and stable indexed. We now have an setup allowing us based on AzureAD-groups => AWS cognito => rule based mapping to AWS IAM roles => usage as "backend roles" in "opensearch role mappings" combined with document-based security, using the "labels.owner" field to specifically limit the acces to documents to the teams they are belonging to, while the cloud- and security governance-teams have access to the whole organization / events from all teams. Unfortunately the setup is pretty specific (and a bit dirty regarding the implementation in the es-loader), so I cannot deliver a PR right now. At least this may help someone else as some kind of idea / inspiration how to expand his aes-siem based setup. Cheers, Alexander