Closed ChrsMark closed 1 month ago
I'd love the idea. I have similar use case. In my company we are saving metrics into files with openTSDB format in PVC in kubernetes. Metrics are automatically picked up by sidecar which transfer it to our observability platform. OTLP metric example:
put accessmanager.envoy.cluster.app-metrics-router-dev.zone.us-west1.us-west1-a.upstream_rq_time.P99 1721726558 93.2466666666666 direction=outbound port=4959 infra=mesh dc=us-west1-a namespace=accessmanager-dev pod_name=accessmanager-celery-flower-7f954746bf-qqmw8
I've tried to use stanza
and build something like filelogreceiver
but unfortunately that lib allows only to pick up the logs. If the otelcol lib will support metrics as well it should be pretty simple to write metric receiver with it.
Tested that with https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.106.0. @djaglowski I'm not 100% sure about the process here but I assume we can close this issue since the initial implementation of the component has been shipped.
thanks a lot for doing this!
The purpose and use-cases of the new component
This new connector will be used in order to extract
otlpjson
from incoming Logs. The connector will be responsible for unpacking valid otlp json from Log records and specifically theBody
field (this can be configurable).This component will serve similarly to the
otlpjsonfilereceiver
but in a later stage in the pipeline. This is particularly useful when theotlpjson
cannot be consumed directly by the aforementioned receiver. An example use-case can be found at https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33846.Example configuration for the component
Initially the configuration can be quite basic like in
otlpjsonfilereceiver
but we can expand later according to the needs.The connector can be used in a pipeline to extract Logs, Metrics and Traces like this:
Telemetry data types supported
1) Logs to Logs 2) Logs to Metrics 3) Logs to Traces
Is this a vendor-specific component?
Code Owner(s)
@djaglowski, @ChrsMark
Sponsor (optional)
@djaglowski (https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33846#issuecomment-2228738273)
Additional context
1) Filing this issue after we shortly chatted about it with @zeitlinger on Slack. 2) A very basic PoC illustrating the idea can be found at https://github.com/ChrsMark/otlpjsonconnector.