search

grafana / helm-charts

Apache License 2.0
1.59k stars 2.24k forks source link

[tempo-distributed] Add OTel gRPC endpoints to nginx gateway #1389

Open AlexDCraig opened 2 years ago

AlexDCraig commented 2 years ago

I'd like to hit the OTLP gRPC endpoints the distributor has. Looks to me like the chart only presently supports the http endpoints for nginx routing.

agermain commented 2 years ago

+1. Sounds like a duplicate of #733.

ahaw023 commented 2 years ago

@AlexDHoffer

It is a little daft

I am using an OpenTelemetry collector and this is the config I set up to make it work

    exporters:
       otlphttp/grafana-hosted:
        endpoint: http://tempo-distributed-gateway.grafana-tempo.svc.cluster.local:80/otlp
ahaw023 commented 2 years ago

longer term fix will be to add a listener for grpc and proxy back to the config

AlexDCraig commented 2 years ago

I don't know about others but I would be perfectly satisfied with circumventing the Tempo gateway pods entirely (i.e. let them just focus on routing http traffic) and simply adding an ingress resource that supports grpc traffic that routes to the distributor. I guess the question with this issue is how would you like to see distributor grpc endpoints exposed.

If what I said above is satisfactory to people I could write it and submit it, I just am pretty uncertain that makes everybody happy.

artyom-p commented 2 years ago

Also struggling with the same issue atm, tried to implement a workaround, the same as mentioned above - otlphttp exporter, but for some reason otlphttp cant send http headers for me, maybe im doing something wrong...

artyom-p commented 2 years ago

Ah, heres why its not working... https://github.com/grafana/tempo/issues/495

sambacha commented 1 year ago

This has been fixed in https://github.com/grafana/tempo/issues/495