getting error [id=222] SEVERE i.o.s.internal.ThrottlingLogger#doLog: Failed to export spans. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180

[aslodh]

Hello Team, i have jenkins installed on AKS kubernetes cluster, jenkins is running as pod I have installed and configure opentelemetry plugin on jenkins and configured as per steps but getting below error while sending metric to jaeger or any other dashboards.

[id=222]   SEVERE  i.o.s.internal.ThrottlingLogger#doLog: Failed to export spans. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180

Any suggention on this?

Thank you in advance

[cyrille-leclerc]

Can you please share more context:

• What's the OTLP destination / which OpenTelemetry backend are you using?
• Can you use an OpenTelemetry Collector next to your Jenkins deployment to ack as a relay to send the traces, logs, and metrics to the OpenTelemetry backend and see if the problem happens again?

[aslodh]

Hi Cyrille,

• What's the OTLP destination / which OpenTelemetry backend are you using? -> We are using jaeger as observibility backend, Also we do have installed jenkins in one namespace and open telemetry collector (otlp) in another namespace) on aks kubernetes cluster

• Can you use an OpenTelemetry Collector next to your Jenkins deployment to ack as a relay to send the traces, logs, and metrics to the OpenTelemetry backend and see if the problem happens again? -> are you saying to install both jenkins and opentelemetry in same namespace? or opentelemetry-java-agent jar as part of jenkins deployment and start jenkins with that opentelemetry jar?

Thank you Ashish

[aslodh]

Hello Cyrille,

I also tried the suggeted thing by you putted jenkins and otel collector in same namespace , but still the issue is same also one more thing to add here when I have added property "otel.exporter.otlp.protocol=http/protobuf" in opentelemetry plugin configure property section the error got changed, if I keep this property I am getting below error

2023-05-23 06:16:45.892+0000 [id=190] SEVERE i.o.s.internal.ThrottlingLogger#doLog: Failed to export spans. The request could not be executed. Full error message: unexpected end of stream on http://px-otel-XXXXXXXXXXXXX.com/...

Thank you Ashish

[cyrille-leclerc]

@aslodh can you confirm:

• you actually need to use http/protobuff rather than the default gRPC protocol?
• if so, have configured the OpenTelemetry Collector to not only listen on the GRPC endpoint (port 4317 by default) but also listen on the http/protobuff port (port 4318 by default). See documentation here
• and have configured the Jenkins OpenTelemetry plugin to send the http/protobuff to this endpoint?

[aslodh]

Hello Cyrille, you actually need to use http/protobuff rather than the default gRPC protocol? -> I tried with both protocols both gives errors, is there any issue if we installed opentelemetry collector on kubernetes and also jenkins on kubernetes (kubernetes version - 1.25.6)

if so, have configured the OpenTelemetry Collector to not only listen on the GRPC endpoint (port 4317 by default) but also listen on the http/protobuff port (port 4318 by default). See documentation here

-> Yes I did tried with both ports by configuring ingress for 4317 and 4318 but still not working

and have configured the Jenkins OpenTelemetry plugin to send the http/protobuff to this endpoint?

• yes, grpc protocol giving error for frame size then I have enabled http/protobuff endpoint and also added configuration property to enable it "otel.exporter.otlp.protocol=http/protobuf"

but both are not working and getting above errors mentioned in ticket

Thank you Ashish

[aslodh]

Hello Cyrille, Any update on above issue?

[kuisathaverat]

It smells like a proxy issue, something like https://stackoverflow.com/questions/63138358/kubernetes-ingress-with-grpc not related to the plugin at all.

[cyrille-leclerc]

Closing this issue as "can't reproduce", this seems to be a Kubernetes setup issue more than a problem caused by the Jenkins OpenTelemetry plugin.