Feature request: Kubernete Events Logging Enrichment

ralongit commented 11 months ago

The purpose and use-cases of the new component

The events receiver provides logging based on the events endpoint and it’s usually missing information like which related workloads and resources are impacted by each event. It also doesn’t contain the old & new objects of the modified resource if it was updated.

I'd like to purpose using a dynamic informer to watch for events from different API groups, process them and enrich them with relevant information by leveraging the Kubernetes API.

It can be added to the existing receivers like k8seventsreceiver, k8sobjectsreceiver with a flag or as a new receiver.

Example use cases:

A 'Secret' kind resource object value was removed and it's being used by many applications. The log should contain the event data with the deleted object content and also have a list of resources and workloads that reference the secret as applications might fail due to the missing resource.
A 'ConfigMap' kind resource object value has changed and it's being used by many applications. The old & new resource object should be in the a log along with a list of related pods, deployments, daemonsets and statefulsets that use it.
A 'Deployment' kind resource object value was added and it's being used to create new pods. The log should have the information about the resource and a list of related configmaps, secrets, service accounts, cluster roles, etc..

Example configuration for the component

receivers:
  k8s_events:
    auth_type: kubeConfig
    namespaces: [default, my_namespace]
    enrich: true

Telemetry data types supported

logs

Is this a vendor-specific component?

[ ] This is a vendor-specific component
[ ] If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

Code Owner(s)

No response

Sponsor (optional)

No response

Additional context

A generic log example of a ConfigMap resource update can be found here.

Note the relatedClusterServices key and the difference in the data key values.

Any feedback on this suggestion will be appreciated.

Other suggestions are welcomed as well.

I will be glad to contribute to this feature and open a PR for it.

JaredTan95 commented 10 months ago

hi @ralongit From the description, this is not a new component, more like a feature enhancement on top of an existing component, like k8seventsreceiver, k8sobjectsreceiver?

ralongit commented 10 months ago

hi @ralongit From the description, this is not a new component, more like a feature enhancement on top of an existing component, like k8seventsreceiver, k8sobjectsreceiver?

@JaredTan95 Yes, the k8sobjectsreceiver seems like a good candidate to have the enhanced data per my described use case.

JaredTan95 commented 10 months ago

@JaredTan95 Yes, the k8sobjectsreceiver seems like a good candidate to have the enhanced data per my described use case.

Ok, then, we'd better update the title or description without New component

github-actions[bot] commented 8 months ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

atoulme commented 7 months ago

@open-telemetry/collector-contrib-triagers would you please help triage this issue?

github-actions[bot] commented 7 months ago

Pinging code owners for receiver/k8sobjects: @dmitryax @hvaghani221 @TylerHelmuth. See Adding Labels via Comments if you do not have permissions to add labels yourself.

ralongit commented 7 months ago

I'd like to add that I've already developed a similar functionality in a separate tool, but I would love to have a more extended version with the same logic within the OpenTelemetry collector. https://github.com/logzio/logzio-k8s-events

github-actions[bot] commented 5 months ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

receiver/k8sobjects: @dmitryax @hvaghani221 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 3 months ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

receiver/k8sobjects: @dmitryax @hvaghani221 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 1 month ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

receiver/k8sobjects: @dmitryax @hvaghani221 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

open-telemetry / opentelemetry-collector-contrib