open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector

https://opentelemetry.io

Apache License 2.0

2.83k stars 2.22k forks source link

New component: container processor #33822

Open h0cheung opened 1 month ago

h0cheung commented 1 month ago

The purpose and use-cases of the new component

There is now a k8sattributes processor' that associates data with k8s metadata. But not for normal containers. So I would like to add such a component. Thek8sattributes processor' could perhaps also be maintained as part of this component.

18920 made a similar request but was eventually closed due to lack of response

Example configuration for the component

similar to k8sattributes processor, with some fields for accessing container APIs:

docker_endpoint: unix:///var/run/docker.sock

Telemetry data types supported

traces, metrics, logs

Is this a vendor-specific component?

[ ] This is a vendor-specific component
[ ] If this is a vendor-specific component, I am a member of the OpenTelemetry organization.
[ ] If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

Code Owner(s)

No response

Sponsor (optional)

No response

Additional context

No response

atoulme commented 1 month ago

Is the resourcedetectionprocessor not good for this? What would this processor set?

h0cheung commented 1 month ago

Is the resourcedetectionprocessor not good for this? What would this processor set?

metadata of docker container, including name, id, labels, etc.

atoulme commented 1 month ago

Maybe worth adding to it: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor#docker-metadata

h0cheung commented 1 month ago

Maybe worth adding to it: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor#docker-metadata

Those metadata could only be got from Docker API. Maybe it's better to use something similar to k8sattributes processor, than to mount the API into containers.

atoulme commented 1 month ago

Sorry, I don't understand. The k8sattributes processor makes requests to the kubernetes API server. The docker resourcedetectionprocessor makes requests to the docker API endpoint. Are you trying to use the kubernetes API server to get container information?

h0cheung commented 1 month ago

Sorry, I don't understand. The k8sattributes processor makes requests to the kubernetes API server. The docker resourcedetectionprocessor makes requests to the docker API endpoint. Are you trying to use the kubernetes API server to get container information?

It uses Docker API. But the resourcedetection processor runs inside a container, which is similar to a sidecar in k8s, and it could only detect the hostname. This processor should run on the host, or a node-level container which is similar to daemonset in k8s. It will receive datas from in-container agents and associate them with all Docker metadata.

atoulme commented 1 month ago

Is the difference that you only get the hostname right now?