[RFC] Add categories to log types

[xeniatup]

The purpose of this RFC (request for comments) is to gather community feedback on a new proposal for log types categorization in Security Analytics plugin.

Problem

Currently the log types for data source are presented as an unstructured list in alphabetical order. Adding custom (user-defined) log types might present a scaling challenges for selecting, filtering, and searching the log types.

Solution

We propose to introduce categorization by grouping log types into logical buckets based on the type of service or application produced the log. This should help with finding specific log types faster and more confidently in the experiences like “Create detector”. Selecting multiple log types of a similar origin will be simplified as they will be grouped together. The categories will help to handle potential increase in the number of log types. Custom (user-defined) log types can be added to any of the categories.

Proposed structure:

Access Management

• AD/LDAP
• Apache access
• Okta

System activity

• System logs (Linux)
• Windows

Network/Endpoint activity

• Network
• DNS
• VPC Flow

Applications

• Microsoft 365
• Google Workspace
• Github

Cloud services

• AWS CloudTrail
• S3 access logs
• Azure

Security findings

• Crowdstrike
• WAF

Other

• For the custom log types that don't belong to any of the categories

Request for comments:

• What do you think about the proposed grouping?
• Do you have any feedback on the naming for the categories?
• Would you organize the log types differently based on your use case? How?
• Do you have any other suggestions on the topic?

[xeniatup]

We should also consider the impact that https://github.com/opensearch-project/security-analytics/issues/573 and https://github.com/opensearch-project/security-analytics/issues/572 will have to the proposed structure.

[praveensameneni]

Adding to backlog