search

open-telemetry / opentelemetry-java-instrumentation

OpenTelemetry auto-instrumentation and instrumentation libraries for Java
https://opentelemetry.io
Apache License 2.0
1.9k stars 827 forks source link

Instrumentation agent directly export the trace to kakfa in otlp format #8137

Closed diracchehn closed 1 year ago

diracchehn commented 1 year ago

Is your feature request related to a problem? Please describe. Currently instrumentation agent will build a keep-alived connection with the collector, if one agent capture too much trace in a certain period, it will lead the instrumentation agent out of memory or the collector has been killed by oom. Especially when the application has been hosted in an on-premise data center and the collector has been hosted on the cloud. If the trace data is relatively big and the network latency is high, it will happen above issues.

Describe the solution you'd like I suggest make the java instrumentation able to directly send trace to Kafka or a message queue like skywalking. It will decouple the agent and the collector and give better performance.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

mateuszrzeszutek commented 1 year ago

Hey @diracchehn ,

Do I understand it correctly that you'd like to have a Java implementation of the collector's Kafka exporter?

diracchehn commented 1 year ago

Yes, a kafka exporter will be great. Meanwhile, the instrumentation shall able to define the topic and some kafka parameters.

trask commented 1 year ago

Meanwhile, the instrumentation shall able to define the topic and some kafka parameters.

can you expand on what you mean by this? thx

github-actions[bot] commented 1 year ago

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed if no further activity occurs within 7 days of this comment.

diracchehn commented 1 year ago

For full stack monitoring, the APM software usually hosted on cloud, but the application needs to be mointored will hosted in DC and other cloud. If the entire applications are in a hybrid cloud environment, the network latency will be high. Usually the java auto-instrumentation will export the trace and metrics to a otel-collector. How ever it consume too much resource on the application side. Just a suggestion, let the agent direct export the trace to a Kafka, then the collector could consume it in a controled frequency, the collector could be in another cloud or in DC. That is my idea, hope I explain it properly.

mateuszrzeszutek commented 1 year ago

Hey @diracchehn , I get your use case -- the problem is, this issue is still to vague to even consider it. Do you want to reimplement the collector's Kafka exporter (that I linked a couple posts above) in Java? Or do you have another ideas about the desired behavior/format/specification of the exporter?

github-actions[bot] commented 1 year ago

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed if no further activity occurs within 7 days of this comment.