Closed aslodh closed 11 months ago
Can you please share more context:
Hi Cyrille,
What's the OTLP destination / which OpenTelemetry backend are you using? -> We are using jaeger as observibility backend, Also we do have installed jenkins in one namespace and open telemetry collector (otlp) in another namespace) on aks kubernetes cluster
Can you use an OpenTelemetry Collector next to your Jenkins deployment to ack as a relay to send the traces, logs, and metrics to the OpenTelemetry backend and see if the problem happens again? -> are you saying to install both jenkins and opentelemetry in same namespace? or opentelemetry-java-agent jar as part of jenkins deployment and start jenkins with that opentelemetry jar?
Thank you Ashish
Hello Cyrille,
I also tried the suggeted thing by you putted jenkins and otel collector in same namespace , but still the issue is same also one more thing to add here when I have added property "otel.exporter.otlp.protocol=http/protobuf" in opentelemetry plugin configure property section the error got changed, if I keep this property I am getting below error
2023-05-23 06:16:45.892+0000 [id=190] SEVERE i.o.s.internal.ThrottlingLogger#doLog: Failed to export spans. The request could not be executed. Full error message: unexpected end of stream on http://px-otel-XXXXXXXXXXXXX.com/...
Thank you Ashish
@aslodh can you confirm:
Hello Cyrille, you actually need to use http/protobuff rather than the default gRPC protocol? -> I tried with both protocols both gives errors, is there any issue if we installed opentelemetry collector on kubernetes and also jenkins on kubernetes (kubernetes version - 1.25.6)
if so, have configured the OpenTelemetry Collector to not only listen on the GRPC endpoint (port 4317 by default) but also listen on the http/protobuff port (port 4318 by default). See documentation here
-> Yes I did tried with both ports by configuring ingress for 4317 and 4318 but still not working
and have configured the Jenkins OpenTelemetry plugin to send the http/protobuff to this endpoint?
yes, grpc protocol giving error for frame size then I have enabled http/protobuff endpoint and also added configuration property to enable it "otel.exporter.otlp.protocol=http/protobuf"
but both are not working and getting above errors mentioned in ticket
Thank you Ashish
Hello Cyrille, Any update on above issue?
It smells like a proxy issue, something like https://stackoverflow.com/questions/63138358/kubernetes-ingress-with-grpc not related to the plugin at all.
Closing this issue as "can't reproduce", this seems to be a Kubernetes setup issue more than a problem caused by the Jenkins OpenTelemetry plugin.
Hello Team, i have jenkins installed on AKS kubernetes cluster, jenkins is running as pod I have installed and configure opentelemetry plugin on jenkins and configured as per steps but getting below error while sending metric to jaeger or any other dashboards.
Any suggention on this?
Thank you in advance