OTLP endpoint: Convert OpenTelemetry resource attributes (`service.namespace`, `service.name` and `service.instance.id`) to Loki `job` and `instance` labels

[F21]

Is your feature request related to a problem? Please describe. OpenTelemetry defines semantic conventions for attribute names. Namely, service.namespace, service.name and service.instance.id. These can be mapped to the instance and job labels used by loki. Prometheus already does this by setting the job label to service.namespace/service.name and the instance label to service.instance.id upon ingestion into the OTLP endpoint. To be consistent, loki should do this mapping for logs ingested via the OTLP endpoint.

Describe the solution you'd like Map service.namespace, service.name and service.instance.id to loki equivalents of instance and job.

Describe alternatives you've considered None

Additional context None

[cyrille-leclerc]

The OTLP endpoint introduced by Loki v3.0 directly promotes the resource attributes you are asking for as Loki labels: service_namespace, service_name and service_instance_id. Is this new capability solving your problem?

Note that the Loki OTLP endpoint doesn't create job and instance labels to connect OpenTelemetry and Prometheus conventions, it is focused on the OpenTelemetry resource attributes.

More details are available on https://grafana.com/docs/loki/latest/send-data/otel/#format-considerations

[F21]

While the new capability makes service_namespace, service_name and service_instance into labels in Loki, it's inconsistent compared to prometheus. This is especially noticeable in Grafana where our queries for Prometheus use job and instance, but for loki queries, we need to context switch to service_namespace, service_name and service_instance. I think it would be a lot more consistent and user friendly if loki can convert the opentelemetry resource attributes to job and instance just like Prometheus.

[cyrille-leclerc]

Thanks for this clear explanation Francis. We have to evaluate this with @stevendungan .