Open adrianchen-es opened 2 months ago
Summary
Add agent.group,host.group field to facilitate searching for a group of agents/hosts based on a user-configurable arbitrary grouping.
agent.group
host.group
Motivation: These arbitrary user-configurable fields will improve the search and analysis experience for Security and Observability.
Security:
Observability:
ABAC:
Once added as an ECS field, it should be included in the Elastic Security fields.
Currently, users create a custom field to facilitate this type of grouping.
Detailed Design: The agent.group or host.group field could treated like data_stream.namespace but used to group agents or hosts instead of data.
data_stream.namespace
Field names
Example values for the fields -APP-A_PROD -APP-A_DEV -APP-B_DEV
APP-A_PROD
APP-A_DEV
APP-B_DEV
Suggested appropriate datatypes Type: keyword
Any example events that map to the proposed use case(s)
Event A from host001 in the APP-A_PROD group
{ "_index": ".ds-metrics-system.process.summary-default-2024.05.03-000042", "_id": "FVwmT7YoA8jJjD7HAAABj0PMPl0", "_version": 1, "_score": 0, "_source": { "@timestamp": "2024-05-04T13:30:09.885Z", "agent": { "ephemeral_id": "85174b4b-4415-46c4-89ce-160e27a56bc5", "id": "a8435320-1d60-49f7-b1e7-25ed5d5eb59c", "name": "host001", "group": "APP-A_PROD", "type": "metricbeat", "version": "8.13.3" }, "data_stream": { "dataset": "system.process.summary", "namespace": "default", "type": "metrics" }, "ecs": { "version": "8.0.0" }, "elastic_agent": { "id": "a8435320-1d60-49f7-b1e7-25ed5d5eb59c", "snapshot": false, "version": "8.13.3" }, "event": { "agent_id_status": "verified", "dataset": "system.process.summary", "duration": 16289873, "ingested": "2024-05-04T13:30:10Z", "module": "system" }, "host": { "architecture": "x86_64", "containerized": false, "hostname": "host001", "group": "APP-A_PROD", "id": "0ba1be1199e74165a458a3bb0f65fb8f", "ip": [ ], "mac": [ ], "name": "host001", "os": { "codename": "Plow", "family": "redhat", "kernel": "5.14.0-427.13.1.el9_4.x86_64", "name": "Red Hat Enterprise Linux", "platform": "rhel", "type": "linux", "version": "9.0 (Plow)" } }, "metricset": { "name": "process_summary", "period": 10000 }, "service": { "type": "system" }, "system": { "process": { "summary": { "disk_sleep": 1, "idle": 96, "sleeping": 272, "threads": { "blocked": 14, "running": 1 }, "total": 370, "zombie": 1 } } } } }
Event B from host002 in the APP-A_PROD group
{ "_index": ".ds-metrics-system.process.summary-default-2024.05.03-000042", "_id": "FVwmT7YoA8jJjD7HAAABj0PMPl0", "_version": 1, "_score": 0, "_source": { "@timestamp": "2024-05-04T13:30:09.885Z", "agent": { "ephemeral_id": "85174b4b-4415-46c4-89ce-160e27a56bc5", "id": "a8435320-1d60-49f7-b1e7-25ed5d5eb59c", "name": "host002", "group": "APP-A_PROD", "type": "metricbeat", "version": "8.13.3" }, "data_stream": { "dataset": "system.process.summary", "namespace": "default", "type": "metrics" }, "ecs": { "version": "8.0.0" }, "elastic_agent": { "id": "a8435320-1d60-49f7-b1e7-25ed5d5eb59c", "snapshot": false, "version": "8.13.3" }, "event": { "agent_id_status": "verified", "dataset": "system.process.summary", "duration": 16289873, "ingested": "2024-05-04T13:30:10Z", "module": "system" }, "host": { "architecture": "x86_64", "containerized": false, "hostname": "host002", "group": "APP-A_PROD", "id": "0ba1be1199e74165a458a3bb0f65fb8f", "ip": [ ], "mac": [ ], "name": "host002", "os": { "codename": "Plow", "family": "redhat", "kernel": "5.14.0-427.13.1.el9_4.x86_64", "name": "Red Hat Enterprise Linux", "platform": "rhel", "type": "linux", "version": "9.0 (Plow)" } }, "metricset": { "name": "process_summary", "period": 10000 }, "service": { "type": "system" }, "system": { "process": { "summary": { "disk_sleep": 1, "idle": 96, "sleeping": 272, "threads": { "blocked": 14, "running": 1 }, "total": 370, "zombie": 1 } } } } }
Summary
Add
agent.group
,host.group
field to facilitate searching for a group of agents/hosts based on a user-configurable arbitrary grouping.Motivation: These arbitrary user-configurable fields will improve the search and analysis experience for Security and Observability.
Security:
agent.group
orhost.group
Observability:
agent.group
orhost.group
ABAC:
agent.group
orhost.group
Once added as an ECS field, it should be included in the Elastic Security fields.
Currently, users create a custom field to facilitate this type of grouping.
Detailed Design: The
agent.group
orhost.group
field could treated likedata_stream.namespace
but used to group agents or hosts instead of data.Field names
agent.group
host.group
Example values for the fields -
APP-A_PROD
-APP-A_DEV
-APP-B_DEV
Suggested appropriate datatypes Type: keyword
Any example events that map to the proposed use case(s)
Event A from host001 in the APP-A_PROD group
Event B from host002 in the APP-A_PROD group