Logstash Azure parser: add GraphAPI log

philhagen / sof-elk

Configuration files for the SOF-ELK VM

GNU General Public License v3.0

1.46k stars 272 forks source link

Logstash Azure parser: add GraphAPI log #282

Closed Pierre450 closed 11 months ago

Pierre450 commented 11 months ago

Can you please update the Azure logstash parser to support the new Graph API log?

Attached is a mapping for the fields. I will provide a log sample separately. GraphAPISchemaMapping.xlsx

philhagen commented 11 months ago

working on it today. can you please confirm the following:

Should these be ingested from /logstash/azure/?
Is checking that the category field is equal to MicrosoftGraphActivityLogs sufficient for uniquely identifying this log type?
Do you have a documentation URL for the schema? (e.g. this for SignIn logs)
Which branch(es) should this capability be merged to? I will put it on public/v20230623 for sure, but wasn't sure if you wanted it to be added to your latest class/v* branch as well.

Pierre450 commented 11 months ago

Yes, the benefit is to correlate this log with other Azure logs, so /logstash/azure to go to the azure-* index would be ideal.
Every log entry has been consistent with the category and I have not seen MicrosoftGraphActivityLogs used anywhere else, so I would say it's sufficient.
for509.com/microsoftgraphactivitylogs is the only published schema documentation I have been able to find for this new log.
For now, l prefer the public branch.

philhagen commented 11 months ago

responseSizeBytes will become destination_bytes to match other similar fields

philhagen commented 11 months ago

do you want roles broken up into an array by splitting e.g. Directory.Read.All User.Read.All Mail.Read Calendars.ReadWrite on the . character?

Pierre450 commented 11 months ago

do you want roles broken up into an array by splitting e.g. Directory.Read.All User.Read.All Mail.Read Calendars.ReadWrite on the . character?

Not needed since that field only indicates the roles associated with the application and not the role "used" for that particular action. The schema has another field called "scope" which had given me hope to have more precise information, but all the sample logs I have only have null in that field.

philhagen commented 11 months ago

looks like the roles are split on a space character and the . is part of the role! https://learn.microsoft.com/en-us/graph/permissions-reference

Pierre450 commented 11 months ago

looks like the roles are split on a space character and the . is part of the role! https://learn.microsoft.com/en-us/graph/permissions-reference

Correct, the permission is Directory.Read.All in its entirety.