OTRF / OSSEM-DM

OSSEM Detection Model
MIT License
167 stars 43 forks source link

Uniquely identify Windows events for automated data collection #55

Open b1t-hunter opened 2 years ago

b1t-hunter commented 2 years ago

Dear OSSEM-DM Team,

thanks for your great work! I have worked with your detection model relationships to extract Windows events that would be required for comprehensive monitoring in a Windows domain. In a past version of your generated relationships (__all_ossemrelationships.yml), you always specified the channel (log_channel), provider (log_source) and event_id for each referenced Windows-based security event. This allowed to generate corresponding log policies and Windows event subscriptions to retrieve those events.

In a newer version of your specification, those fields have a changed semantics (log_source) or have been removed (log_channel). This makes the automated derivation of required events much more difficult or even impossible. Is it planned that this information is reintroduced, maybe under a different key name? If not, do you see any other way to retrieve the channel and provider from the relationship data, maybe with some mapping table?

While the simple cases of Microsoft-Windows-Security-Auditing and sysmon could be mapped to the channel/provider-pairs Security:Microsoft-Windows-Security-Auditing and Microsoft-Windows-Sysmon/Operational:Microsoft-Windows-Sysmon, this becomes much more difficult for event providers that log into multiple channels, such as Microsoft-Windows-Eventlog. So in order to derive the corresponding WEF subscription or generally event filter, I would have to know which event_id belongs to which channel/provider pair. On the other hand, there are some log sources that go into two different providers (and channels), like Powershell. Powershell seems to be covered now by the two log_sources powershelland Microsoft-Windows-PowerShell. powershell seems to be used for the two providers PowerShell (channel: Windows PowerShell) and Microsoft-Windows-PowerShell (channel: Microsoft-Windows-PowerShell/Operational). Hence, it cannot be determined automatically (or manually without additional knowledge of existing events) whether events should be collected from either Windows PowerShell or Microsoft-Windows-PowerShell/Operational channel.

Cyb3rPandaH commented 2 years ago

Hey @b1t-hunter , I hope you are doing well πŸ˜ƒ

We are happy to hear you are using the project in your environment. Also, thank you for taking the time to submit this issue.

Regarding renaming of fields (provider --> log_source): we decided to use a more general term when identifying the source of telemetry collected. This is helping us to continue documenting the project with telemetry from different platforms in a more standard way. So the equivalent to provider would be log_source now.

Regarding the log_channel field: we decided to remove the field to align the schemas of OSSEM Data Dictionaries (DD) and Detection Modeling (DM). The log_channel field was Windows focused and we did not consider it as part of the key (log_source / id / version / platform) to identify a data dictionary uniquely within DD. However, the use case that you describe here is very interesting. We are already using optional / conditional fields within DM such as audit_category and audit_sub_category that are related to the Windows platform, but let me get some feedback from the team first.

Then we can make a final decision about adding the "channel" field to the yaml schema.

Thank you again 🍻

Cyb3rPandaH commented 2 years ago

Some notes to consider:

b1t-hunter commented 2 years ago

Hi @Cyb3rPandaH,

that sounds great! πŸ˜ƒ I like what you are doing in this project, because it is more free and beyond what the ATT&CK datamodels are providing at the moment. Especially when it comes to the standardization of event specs. And it seems you are contributing the experience you get from this project into ATT&CK datamodels.

I saw that you have introduced audit_category and audit_sub_category, which greatly helps to derive an audit policy automatically. At the moment, I do this derivation based on the event code, if it is part of the Microsoft-Windows-Security-Auditing provider. So that could help to skip that extra lookup step.

Cyb3rPandaH commented 2 years ago

Hey @b1t-hunter , hope you are doing well πŸ˜ƒ

We have decided to add the channel field to our YAML schema. I have updated the README file of the repo (Developer branch) with an initial description of the YAML schema (Including channel). I will really appreciate it if you can take a look at it and let me know if you have any comment or feedback: https://github.com/OTRF/OSSEM-DM/blob/developer/README.md

I will update all theYAML files so they include the channel field over this week and push all the changes to the main branch by the end of this weekend.

Thank you 🍻

b1t-hunter commented 2 years ago

Thanks for your support, that's great news! πŸ˜ƒ That should make the matching much easier again.

I think the current schema looks nice. I saw that you already fixed the sysmon parts to Microsoft-Windows-Sysmon. That means you would also switch powershell to the corresponding Microsoft-Windows-Powershell and PowerShell providers?

Cyb3rPandaH commented 2 years ago

Thank you @b1t-hunter

and yes, I will update Security Auditing, System and PowerShell logs too

Cyb3rPandaH commented 2 years ago

Thank you for the feedback and suggestions @b1t-hunter 🍻 I just merged a PR to the main branch that closes this issue. We have added the channel field for the following providers: Security Auditing, Sysmon, PowerShell, Windows Firewall, Service Control Manager and Event Log. Please let me know in case you have any comments or feedback πŸ™

b1t-hunter commented 2 years ago

Thanks for implementing those changes and for fixing all the channels πŸ˜ƒ . Automation can go on now 🦾

b1t-hunter commented 2 years ago

Hi @Cyb3rPandaH,

I have checked the current config with my old scripts, and works nicely for the majority of events, but there are still a few that are missing channel information:

File event_id channel log_source
sensor_health_changed 1100 Security βœ”οΈ
sensor_health_changed 1101 Security βœ”οΈ
sensor_health_changed 1102 Security βœ”οΈ
sensor_health_changed 1104 Security βœ”οΈ
user_created_wmi_object 5857 Microsoft-Windows-WMI-Activity/Operational βœ”οΈ
user_created_wmi_object 5858 Microsoft-Windows-WMI-Activity/Operational βœ”οΈ
user_created_wmi_object 5859 Microsoft-Windows-WMI-Activity/Operational βœ”οΈ
user_created_wmi_object 5860 Microsoft-Windows-WMI-Activity/Operational βœ”οΈ
user_created_wmi_object 5861 Microsoft-Windows-WMI-Activity/Operational βœ”οΈ
user_created_wmi_object 5861 Microsoft-Windows-WMI-Activity/Operational βœ”οΈ
wmi_object_created 5861 Microsoft-Windows-WMI-Activity/Operational βœ”οΈ
process_searched_ldap 30 Microsoft-Windows-LDAP-Client/Debug Microsoft-Windows-LDAP-Client
Cyb3rPandaH commented 2 years ago

Hey @b1t-hunter

Sorry for the late reply. Thank you for letting me know about this. I will check those relationships over the weekend and update them πŸ‘