elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.06k stars 4.89k forks source link

[Agent] Allow users to add custom metadata on fleet-managed agents #38671

Open yehlo opened 3 months ago

yehlo commented 3 months ago

Describe the enhancement: We are planning to switch to managed agent compared to unmanaged but would need the ability to add custom metadata for all configured integrations.

It should be possible somewhat like the beats being able to just add custom fields as fields.my_custom1: my_value1 in a managed agent installation aswell.

These fields should then be appended to all documents by the agent to make them searchable.

Enrolling the agent with the --tag option doesn't solve this issue, because this will be added in fleet ui only.

Doing this through fleet-ui is not an option, because this would mean that already available system metadata will have to be added into the agent-policy aswell, making it super cumbersome + inconvenient because we would need one policy with corresponding processors to add this data.

Describe a specific use case for the enhancement or feature: Imagine you're running 10-20k systems with fleet-managed agent where each customer should be able to see the data of his own system. You want both your system admins be able to add some default fields needed for classification of the data like fields.customer_id: abcd aswell as allow the users to add their own custom fields for further filtering and dashboarding within kibana like fields.my_app: is_awesome .

Without this feature you will produce a lot of duplicate agent-policies with some minor processors added for this.

botelastic[bot] commented 3 months ago

This issue doesn't have a Team:<team> label.