search

tsukada-ascade / spark-structed-streaming-poc

MIT License
0 stars 0 forks source link

Need help with manual OTEL integration for spark jobs #1

Closed himanshujeena closed 8 months ago

himanshujeena commented 8 months ago

Hi Tsukada,

I am trying to instrument our spark structured streaming jobs and came across issue raised by you https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/9638.

I am still going through the documentation on how to manually instrument the job. If possible, please share the boilerplate implementation.

Thanks !

tsukada-ascade commented 8 months ago

Hi himanshujeena,

What you need is an example written in Scala? If I create something like this poc with manual instrumentation in Scala, can it help you?　

himanshujeena commented 8 months ago

Yes, I am looking for Scala example. That would be a great help. Thanks!

tsukada-ascade commented 8 months ago

Hi himanshujeena,

It's done. Please see 'manual' branch.
https://github.com/tsukada-ascade/spark-structed-streaming-poc/tree/manual

Run it, you will get a result like shown below. The "THIS IS MANUAL" span is manual.

Thanks

2024-03-07-spark-poc-manual

himanshujeena commented 8 months ago

Thanks a bunch tsukada :)