Stagemonitor with Zipkin

stagemonitor / stagemonitor-mailinglist

GitHub issues abused as a mailing list

3 stars 0 forks source link

Stagemonitor with Zipkin #76

Open mywoodstock opened 5 years ago

mywoodstock commented 5 years ago

Hello! I am trying to use stagemonitor to trace my java applications and send the traces to Zipkin. I already have Zipkin working for some other monitoring. I added dependencies for stagemonitor-core, stagemonitor-tracing (0.88.4) and stagemonitor-tracing-zipkin (0.80.0.RC1). I execute the application with -javaagent pointing to byte-buddy-agent-1.7.9.jar (although I tried without it as well with the exact same result). I am using JDK8. My stagemonitor.properties looks like: stagemonitor.active=true stagemonitor.profiler.active=true stagemonitor.zipkin.reporter.endpoint=https://<my zipkin server>/api/v2/spans stagemonitor.applicationName=myanalytics

Following is the stagemonitor part of stdout: 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.Stagemonitor - Measurement Session is initialized: [application=myanalytics] [instance=My Instance] [host=workspace-abhinav-2] 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.CorePlugin - Not sending metrics to InfluxDB (url=null, interval=60s) 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.Stagemonitor - # stagemonitor status 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.Stagemonitor - System information: Java 1.8.0_144 (Oracle Corporation) Linux 4.4.0-87-generic 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.Stagemonitor - OK - Agent attachment 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.Stagemonitor - OK - CorePlugin (version 0.88.4) 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.Stagemonitor - OK - Startup 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - # stagemonitor configuration, listing non-default values: 2018-09-06 18:24:28 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.applicationName: myanalytics (source: stagemonitor.properties)

No traces/data are being generated (nothing gets sent to the zipkin server). I tried using the logging option stagemonitor.tracing.reporting.log=true as well, but no change in output.

What am I missing? Any help is appreciated.

Thanks!

mywoodstock commented 5 years ago

After a lot of fiddling, I am able to get the plugin initialized successfully, but it is still not generating any spans (i.e. not sending any data to zipkin server). Following is the stagemonitor output:

INFO org.stagemonitor.tracing.soap.SoapHandlerTransformer - Adding SOAPHandlers [org.stagemonitor.tracing.soap.TracingServerSOAPHandler@7fde7290, org.stagemonitor.tracing.soap.TracingClientSOAPHandler@29c0d4f0]
INFO org.stagemonitor.core.Stagemonitor - Measurement Session is initialized: [application=MyApplication] [instance=MyInstance] [host=MyHost]
INFO org.stagemonitor.core.CorePlugin - Not sending metrics to InfluxDB (url=null, interval=60s)
WARN org.stagemonitor.core.StagemonitorPlugin - The plugin ZipkinPlugin has not been initialized yet. You should define a dependency via StagemonitorPlugin#dependsOn to InitArguments
INFO org.stagemonitor.core.Stagemonitor - # stagemonitor status
INFO org.stagemonitor.core.Stagemonitor - System information: Java 1.8.0_91 (Oracle Corporation) Mac OS X 10.13.6
INFO org.stagemonitor.core.Stagemonitor - OK   - Agent attachment
INFO org.stagemonitor.core.Stagemonitor - OK   - CorePlugin (version 0.88.4)
INFO org.stagemonitor.core.Stagemonitor - OK   - SoapTracingPlugin (version 0.88.4)
INFO org.stagemonitor.core.Stagemonitor - OK   - Startup
INFO org.stagemonitor.core.Stagemonitor - OK   - TracingPlugin (version 0.88.4)
INFO org.stagemonitor.core.Stagemonitor - OK   - ZipkinPlugin (version 0.80.0.RC1)
INFO org.stagemonitor.core.configuration.ConfigurationLogger - # stagemonitor configuration, listing non-default values:
INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.tracing.reporting.log: true (source: stagemonitor.properties)
INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.reporting.interval.console: 1 (source: stagemonitor.properties)
INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.applicationName: Analytics (source: stagemonitor.properties)
INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.instanceName: OrderAppender (source: stagemonitor.properties)
INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.instrument.include: com.onera.analytics (source: stagemonitor.properties)
INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.zipkin.reporter.endpoint: https://<my zipkin server>/api/v2/spans/ (source: stagemonitor.properties)

codefromthecrypt commented 5 years ago

stagemonitor is still using an old version of zipkin libraries. try changing the url to v1

https://github.com/stagemonitor/boot-zipkin/blob/master/src/main/resources/stagemonitor-client.properties On Fri, Sep 14, 2018 at 3:19 AM Abhinav Sarje notifications@github.com wrote:

After a lot of fiddling, I am able to get the plugin initialized successfully, but it is still not generating any spans (i.e. not sending any data to zipkin server). Following is the stagemonitor output:

INFO org.stagemonitor.tracing.soap.SoapHandlerTransformer - Adding SOAPHandlers [org.stagemonitor.tracing.soap.TracingServerSOAPHandler@7fde7290, org.stagemonitor.tracing.soap.TracingClientSOAPHandler@29c0d4f0] INFO org.stagemonitor.core.Stagemonitor - Measurement Session is initialized: [application=MyApplication] [instance=MyInstance] [host=MyHost] INFO org.stagemonitor.core.CorePlugin - Not sending metrics to InfluxDB (url=null, interval=60s) WARN org.stagemonitor.core.StagemonitorPlugin - The plugin ZipkinPlugin has not been initialized yet. You should define a dependency via StagemonitorPlugin#dependsOn to InitArguments INFO org.stagemonitor.core.Stagemonitor - # stagemonitor status INFO org.stagemonitor.core.Stagemonitor - System information: Java 1.8.0_91 (Oracle Corporation) Mac OS X 10.13.6 INFO org.stagemonitor.core.Stagemonitor - OK - Agent attachment INFO org.stagemonitor.core.Stagemonitor - OK - CorePlugin (version 0.88.4) INFO org.stagemonitor.core.Stagemonitor - OK - SoapTracingPlugin (version 0.88.4) INFO org.stagemonitor.core.Stagemonitor - OK - Startup INFO org.stagemonitor.core.Stagemonitor - OK - TracingPlugin (version 0.88.4) INFO org.stagemonitor.core.Stagemonitor - OK - ZipkinPlugin (version 0.80.0.RC1) INFO org.stagemonitor.core.configuration.ConfigurationLogger - # stagemonitor configuration, listing non-default values: INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.tracing.reporting.log: true (source: stagemonitor.properties) INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.reporting.interval.console: 1 (source: stagemonitor.properties) INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.applicationName: Analytics (source: stagemonitor.properties) INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.instanceName: OrderAppender (source: stagemonitor.properties) INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.instrument.include: com.onera.analytics (source: stagemonitor.properties) INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.zipkin.reporter.endpoint: https:///api/v2/spans/ (source: stagemonitor.properties)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

mywoodstock commented 5 years ago

I tried two approaches:

Using v1 in the zipkin api endpoint. This had absolutely no effect, and looking at logs on the zipkin server, it never receives any request from stagemonitor.
I cloned and updated stagemonitor to use zipkin v2. But the result is the same -> still nothing being sent to the zipkin server.

Any ideas?

codefromthecrypt commented 5 years ago

can you try the boot-zipkin demo that I linked above and let me know if it still works?

mywoodstock commented 5 years ago

I tried the example app. The app runs fine, but still no data being sent to zipkin. I tested with both an existing zipkin server, as well as a new test server locally. Also, I had to change the dependency in the pom.xml from stagemonitor-zipkin to stagemonitor-tracing-zipkin. Any ideas on how to approach debugging this? Thanks in advance!

codefromthecrypt commented 5 years ago

ok if the example doesn't work that helps as it is easier for me to validate

On Tue, Sep 18, 2018 at 7:56 AM Abhinav Sarje notifications@github.com wrote:

I tried the example app. The app runs fine, but still no data being sent to zipkin. I tested with both an existing zipkin server, as well as a new test server locally. Also, I had to change the dependency in the pom.xml from stagemonitor-zipkin to stagemonitor-tracing-zipkin. Any ideas on how to approach debugging this? Thanks in advance!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stagemonitor/stagemonitor-mailinglist/issues/76#issuecomment-422208524, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD615hkdINptz9miR4ZwYo7NZCdpDv7ks5ucDadgaJpZM4Wde72 .

mywoodstock commented 5 years ago

Some more tinkering with stagemonitor, after updating to zipkin v2, I get the following error:

java.lang.IllegalArgumentException: Span must be an instance of brave.opentracing.BraveSpan, but was class org.stagemonitor.tracing.wrapper.SpanWrapper
    at brave.opentracing.BraveScopeManager.activate (BraveScopeManager.java:80)
    at brave.opentracing.BraveScopeManager.activate (BraveScopeManager.java:26)
    at org.stagemonitor.tracing.wrapper.SpanWrappingTracer$SpanWrappingSpanBuilder.startActive (SpanWrappingTracer.java:176)
    at org.stagemonitor.tracing.MonitoredMethodRequest.createScope (MonitoredMethodRequest.java:59)
    at org.stagemonitor.tracing.RequestMonitor.monitorStart (RequestMonitor.java:35)
    at org.stagemonitor.tracing.AbstractTracingTransformer.monitorStart (AbstractTracingTransformer.java:46)
    at com.company.Main.execute (Main.java:10)
    at com.company.Main.main (Main.java:40)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:293)
    at java.lang.Thread.run (Thread.java:745)

Looking into the code, BraveScopeManager.activate requires the argument span to be of type BraveSpan, but the span wrapper in stagemonitor creates an instance of SpanWrapper.

Did I miss something?

codefromthecrypt commented 5 years ago

stage monitor is pinned to an old version of opentracing something originally due to jaeger but for a long time they have upgraded. probably best thing is to offer a pull request to stage monitor using latest opentracing jaeger and brave.

On Wed, 19 Sep 2018, 03:56 Abhinav Sarje, notifications@github.com wrote:

Some more tinkering with stagemonitor, after updating to zipkin v2, I get the following error:
at brave.opentracing.BraveScopeManager.activate (BraveScopeManager.java:80)
at brave.opentracing.BraveScopeManager.activate (BraveScopeManager.java:26)
at org.stagemonitor.tracing.wrapper.SpanWrappingTracer$SpanWrappingSpanBuilder.startActive (SpanWrappingTracer.java:176)
at org.stagemonitor.tracing.MonitoredMethodRequest.createScope (MonitoredMethodRequest.java:59)
at org.stagemonitor.tracing.RequestMonitor.monitorStart (RequestMonitor.java:35)
at org.stagemonitor.tracing.AbstractTracingTransformer.monitorStart (AbstractTracingTransformer.java:46)
at com.company.Main.execute (Main.java:10)
at com.company.Main.main (Main.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:293)
at java.lang.Thread.run (Thread.java:745)```
Looking into the code, BraveScopeManager.activate requires the argument span to be of type BraveSpan, but the span wrapper in stagemonitor creates an instance of SpanWrapper.

Did I miss something?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stagemonitor/stagemonitor-mailinglist/issues/76#issuecomment-422529528, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD617DTz03X2soli03PM9g68RZ8VL1Yks5ucU_4gaJpZM4Wde72 .

codefromthecrypt commented 5 years ago

reason I suggest this is even if we get past whatever this is about you later will run into old library issues until the agent is updated

On Wed, 19 Sep 2018, 07:51 Adrian Cole, adrian.f.cole@gmail.com wrote:

stage monitor is pinned to an old version of opentracing something originally due to jaeger but for a long time they have upgraded. probably best thing is to offer a pull request to stage monitor using latest opentracing jaeger and brave.

On Wed, 19 Sep 2018, 03:56 Abhinav Sarje, notifications@github.com wrote:
Some more tinkering with stagemonitor, after updating to zipkin v2, I get the following error:
at brave.opentracing.BraveScopeManager.activate (BraveScopeManager.java:80)
at brave.opentracing.BraveScopeManager.activate (BraveScopeManager.java:26)
at org.stagemonitor.tracing.wrapper.SpanWrappingTracer$SpanWrappingSpanBuilder.startActive (SpanWrappingTracer.java:176)
at org.stagemonitor.tracing.MonitoredMethodRequest.createScope (MonitoredMethodRequest.java:59)
at org.stagemonitor.tracing.RequestMonitor.monitorStart (RequestMonitor.java:35)
at org.stagemonitor.tracing.AbstractTracingTransformer.monitorStart (AbstractTracingTransformer.java:46)
at com.company.Main.execute (Main.java:10)
at com.company.Main.main (Main.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:293)
at java.lang.Thread.run (Thread.java:745)```
Looking into the code, BraveScopeManager.activate requires the argument span to be of type BraveSpan, but the span wrapper in stagemonitor creates an instance of SpanWrapper.

Did I miss something?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stagemonitor/stagemonitor-mailinglist/issues/76#issuecomment-422529528, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD617DTz03X2soli03PM9g68RZ8VL1Yks5ucU_4gaJpZM4Wde72 .