Java Agent for OpenTracing
NOTE: It is deprecated now. Please migrate to OpenTelemetry Instrumentation.
Automatically instruments 3rd-party libraries in Java applications
[
What is SpecialAgent?
The SpecialAgent is software that attaches to Java applications, and automatically instruments 3rd-party libraries within. The SpecialAgent uses the OpenTracing API for Integrations that instrument 3rd-party libraries, as well as Trace Exporters that export trace data to OpenTracing Tracer vendors. The architecture of SpecialAgent was specifically designed to include contributions from the community, whereby its platform automates the installation of OpenTracing Integrations written by individual contributors. In addition to Integrations, the SpecialAgent also supports Trace Exporters, which connect an instrumented runtime to OpenTracing-compliant tracer vendors, such as [LightStep][lightstep], [Wavefront][wavefront], or [Jaeger][jaeger]. Both the Integrations and the Trace Exporters are decoupled from SpecialAgent -- i.e. neither need to know about SpecialAgent. At its core, the SpecialAgent is itself nothing more than an engine that abstracts the functionality for the automatic installation of Integrations, and their connection to Trace Exporters. A benefit of this approach is that the SpecialAgent intrinsically embodies and encourages community involvement.
Both the Integrations and the Trace Exporters are open-source, and are developed and supported by the OpenTracing community.
The SpecialAgent supports Oracle Java and OpenJDK.
Table of Contents
1 [Introduction](#1-introduction)2 [Quick Start](#2-quick-start)
2.1 [Installation](#21-installation)
2.1.1 [In Application](#211-in-application)
2.1.1.1 [Stable](#2111-stable)
2.1.1.2 [Development](#2112-development)
2.1.2 [For Development](#212-for-development)
2.1.2.1 [Integrations](#2121-integrations)
2.1.2.1.1 [Uncoupled Integrations](#21211-uncoupled-integrations)
2.1.2.1.2 [Coupled Integrations](#21212-coupled-integrations)
2.1.2.1.3 [Development of Integration Rules](#21213-development-of-integration-rules)
2.1.2.2 [Trace Exporters](#2122-trace-exporters)
2.1.2.2.1 [Short Name](#21221-short-name)
2.2 [Usage](#22-usage)
2.2.1 [Static Attach](#221-static-attach)
2.2.2 [Dynamic Attach](#222-dynamic-attach)
2.2.3 [Static Deferred Attach](#223-static-deferred-attach)
3 [Configuration](#3-configuration)
3.1 [Overview](#31-overview)
3.2 [Properties](#32-properties)
3.2.1 [Logging](#321-logging)
3.2.2 [Integration](#322-integration)
3.2.3 [General](#323-general)
3.3 [Selecting the Trace Exporter](#33-selecting-the-trace-exporter)
3.4 [Disabling Integration Rules](#34-disabling-integration-rules)
3.4.1 [Disabling All Integration Rules](#341-disabling-all-integration-rules)
3.4.2 [Disabling (or enabling) One Integration Rule](#342-disabling-or-enabling-one-integration-rule)
3.4.3 [Disabling `AgentRule`s of an Integration Rule](#343-disabling-agentrules-of-an-integration-rule)
3.5 [Disabling Trace Exporters](#35-disabling-trace-exporters)
3.6 [Including custom Integration Rules](#36-including-custom-integration-rules)
3.7 [Rewritable Tracer](#37-rewritable-tracer)
4 [Supported Integrations and Trace Exporters](#4-supported-integrations-and-trace-exporters)
4.1 [Integrations](#41-integrations)
4.2 [Trace Exporters](#42-trace-exporters)
4.3 [Libraries instrumented via other Integrations](#43-libraries-instrumented-via-other-integrations)
5 [Objectives](#5-objectives)
5.1 [Goals](#51-goals)
5.2 [Non-Goals](#52-non-goals)
6 [Definitions](#6-definitions)
6.1 [Tracer](#61-tracer)
6.2 [Trace Exporter](#62-trace-exporter)
6.3 [Integration](#63-integration)
6.4 [Integration Rule](#64-integration-rule)
7 [Credits](#7-credits)
8 [Contributing](#8-contributing)
9 [License](#9-license) ## 1 Introduction This file contains the operational instructions for the use and development of SpecialAgent. ## 2 Quick Start The SpecialAgent is a Java Agent that attaches to an application (either [statically or dynamically](#22-usage)). Once attached, the SpecialAgent it loads its bundled [Integrations](#63-integration), which are all enabled by default. The SpecialAgent is stable -- any exception that occurs during attachment of SpecialAgent will not adversely affect the stability of the target application. It is, however, important to note that SpecialAgent bundles [Integrations](#63-integration) that are developed by 3rd parties and individual contributors. We strive to assert the stability of SpecialAgent with rigorous [integration tests][circleci], yet it is still possible that the code in a bundled [Integration](#63-integration) may result in an exception that is not properly handled, which could potentially destabilize a target application. ### 2.1 Installation The Maven build of the SpecialAgent project generates 2 artifacts: **main** and **test**. These artifacts can be obtained by downloading directly from [Maven's Central Repository](https://repo1.maven.org/maven2/io/opentracing/contrib/specialagent/opentracing-specialagent/1.7.4/), or by cloning this repository and following the [Development Instructions](#212-for-development). #### 2.1.1 In Application The SpecialAgent is contained in a single JAR file. This JAR file is the **main** artifact built by Maven, and bundles the [Integrations](#63-integration) from [opentracing-contrib][opentracing-contrib] for which [Integration Rules](#64-integration-rule) have been implemented. To use the SpecialAgent on an application, please download the [stable](#2111-stable) or [development](#2112-development) **main** artifact. The artifact JAR can be provided to an application with the `-javaagent:${SPECIAL_AGENT_JAR}` vm argument for [Static Attach](#221-static-attach) and [Static Deferred Attach](#223-static-deferred-attach). The artifact JAR can also be executed in standalone fashion, which requires an argument to be passed for the PID of a target process to which SpecialAgent is to [dynamically attach](#222-dynamic-attach). Please refer to [Usage](#22-usage) section for usage instructions. ##### 2.1.1.1 Stable The latest stable release is: [1.7.4][main-release] ```bash wget -O opentracing-specialagent-1.7.4.jar "https://repo1.maven.org/maven2/io/opentracing/contrib/specialagent/opentracing-specialagent/1.7.4/opentracing-specialagent-1.7.4.jar" ``` ##### 2.1.1.2 Development The latest development release is: [1.7.5-SNAPSHOT][main-snapshot] ```bash wget -O opentracing-specialagent-1.7.5-SNAPSHOT.jar "https://oss.sonatype.org/service/local/artifact/maven/redirect?r=snapshots&g=io.opentracing.contrib.specialagent&a=opentracing-specialagent&v=LATEST" ``` **Note**: Sometimes the web service call (in the line above) to retrieve the latest SNAPSHOT build fails to deliver the correct download. In order to work around this issue, please consider using the following command (for Linux and Mac OS): ```bash wget -O opentracing-specialagent-1.7.5-SNAPSHOT.jar $(curl -s https://oss.sonatype.org/content/repositories/snapshots/io/opentracing/contrib/specialagent/opentracing-specialagent/1.7.5-SNAPSHOT/ | grep '".*\d\.jar"' | tail -1 | awk -F\" '{print $2}') ``` #### 2.1.2 For Development The SpecialAgent is built in 2 passes utilizing different profiles: 1. The `default` profile is used for development of [Integration Rules](#64-integration-rule). It builds and runs tests for each rule, but _does not bundle the rules_ into the main JAR (i.e. `opentracing-specialagent-1.7.5-SNAPSHOT.jar`). To run this profile: ```bash mvn clean install ``` _**Note**: If you skip tests, the `assemble` profile will display an error stating that tests have not been run. See [Convenient One-Liners](#convenient-one-liners) for quick ways to build and package SpecialAgent_. 1. The `assemble` profile is used to bundle the [Integrations](#64-integration-rule) into the main JAR (i.e. `opentracing-specialagent-1.7.5-SNAPSHOT.jar`). It builds each rule, but _does not run tests._ Once the build with the `assemble` profile is finished, the main JAR (i.e. `opentracing-specialagent-1.7.5-SNAPSHOT.jar`) will contain the built rules inside it. _**Note**: If you do not run this step, the `opentracing-specialagent-1.7.5-SNAPSHOT.jar` from the previous step will not contain any [Integrations](#63-integration)!_ _**Note**: It is important to **not** run Maven's `clean` lifecycle when executing the `assemble` profile, otherwise the [Integrations](#63-integration) built in with the `default` profile will be cleared._ To run this profile: ```bash mvn -Dassemble install ``` * For a one-line build command to build SpecialAgent, its rules, run all tests, and create the `assemble` package: ```bash mvn clean install && mvn -Dassemble install ``` ##### Convenient One-Liners 1. Skipping tests when building SpecialAgent. ```bash mvn -DskipTests clean install ``` 1. Skipping compatibility tests when building SpecialAgent rules. ```bash mvn -DskipCompatibilityTests clean install ``` 1. Packaging SpecialAgent with rules that skipped test execution. ```bash mvn -Dassemble -DignoreMissingTestManifest install ``` ##### 2.1.2.1 [Integrations](#63-integration) The SpecialAgent supports two kinds of [Integrations](#63-integration): ###### 2.1.2.1.1 Uncoupled [Integrations](#63-integration) Uncoupled [Integrations](#63-integration) are those that can be used _without_ SpecialAgent. These [Integrations](#63-integration) are not coupled to SpecialAgent, and can be used via **manual integration** in an application. Uncoupled [Integrations](#63-integration) are implemented in [opentracing-contrib][opentracing-contrib], and do not know about SpecialAgent. To support Uncoupled [Integrations](#63-integration), SpecialAgent requires the implementation of an [Integration Rule](#64-integration-rule) that bridges the Uncoupled [Integrations](#63-integration) to SpecialAgent's auto-instrumentation mechanism. The implementation of [Integrations](#63-integration) as uncoupled is preferred, as this pattern allows users to instrument their applications manually, if so desired. However, not all 3rd-party libraries can be instrumented to allow manual integration, leaving the alternative pattern: Coupled [Integrations](#63-integration). ###### 2.1.2.1.2 Coupled [Integrations](#63-integration) Coupled [Integrations](#63-integration) are those that _can only be used with_ SpecialAgent. These [Integrations](#63-integration) are coupled to SpecialAgent, and can only be used via **automatic installation** in an application. Coupled [Integrations](#63-integration) are effectively [Integration Rules](#64-integration-rule) that implement the full scope of the instrumentation of the 3rd-party library, and directly bridge this integration into the SpecialAgent's auto-instrumentation mechanism. The implementation of [Integrations](#63-integration) as coupled is _discouraged_, as this pattern prohibits users from instrumenting their applications manually, if so desired. However, not all 3rd-party libraries can be instrumented to allow manual integration, leaving Coupled [Integrations](#63-integration) as the only option. ###### 2.1.2.1.3 Development of [Integration Rules](#64-integration-rule) For development of [Integration Rules](#64-integration-rule), import the `opentracing-specialagent-api` and `test-jar` of the `opentracing-specialagent`. ```xml
Commands to Attach | Plugin and Rule
Initialization Timeline | |:-|:-:|:-:| | [Static Attach](#221-static-attach)
| 1 (sync)
| Before app start
(any application) | | [Dynamic Attach](#222-dynamic-attach)
| 2 (async)
| After app start
(any application) | | [Static Deferred Attach](#223-static-deferred-attach)
| 1 (sync)
| After app start
([some applications](#static-deferred-attach-is-currently-supported-for)) | #### 2.2.1 Static Attach With [Static Attach](#221-static-attach), the application is executed with the `-javaagent` argument, and the agent initialization occurs before the application is started. This mode requires 1 command from the command line. Statically attaching to a Java application involves the use of the `-javaagent` vm argument at the time of startup of the target Java application. The following command can be used as an example: ```bash java -javaagent:opentracing-specialagent-1.7.4.jar -jar MyApp.jar ``` This command statically attaches SpecialAgent to the application in `MyApp.jar`. #### 2.2.2 Dynamic Attach With [Dynamic Attach](#222-dynamic-attach), the application is allowed to start first, afterwhich an agent VM is dynamically attached to the application's PID. This mode requires 2 commands from the command line: the first for the application, and the second for the agent VM. Dynamically attaching to a Java application involves the use of a running application’s PID, after the application’s startup. The following commands can be used as an example: 1. To obtain the `PID` of the target application: ```bash jps ``` 1. To attach to the target `PID`: * For jdk1.8 ```bash java -Xbootclasspath/a:$JAVA_HOME/lib/tools.jar -jar opentracing-specialagent-1.7.4.jar ${PID} ``` * For jdk9+ ```bash java -jar opentracing-specialagent-1.7.4.jar ${PID} ``` **Note:** Properties that are provided in the command to dynamically attach will be absorbed by the target application. This applies to properties specific to SpecialAgent, such as `-Dsa.log.level=FINER`, as well as other properties such as `-Djava.util.logging.config.file=out.log`. **Troubleshooting:** If you encounter an exception stating `Unable to open socket file`, make sure the attaching VM is executed with the same permissions as the target VM. #### 2.2.3 Static Deferred Attach With Static Deferred Attach, the application is executed with the `-javaagent` argument, but the agent initialization is deferred until the application is started. This mode requires 1 command from the command line, and is designed specifically for runtimes that have complex initialization lifecycles that may result in extraneously lengthy startup times when attached with [Static Attach](#221-static-attach). ##### Static Deferred Attach is currently supported for: 1. Spring WebMVC (1.0 to LATEST). 1. Spring Boot (1.0.0.RELEASE to LATEST). If the above supported application environment is detected, Static Deferred Attach is automatically activated. **To deactivate** Static Deferred Attach, specify the following system property on the command line: ```bash -Dsa.init.defer=false ``` The following command can be used as an example: ```bash java -javaagent:opentracing-specialagent-1.7.4.jar -Dsa.init.defer=false -jar MySpringBootApp.jar ``` ## 3 Configuration ### 3.1 Overview The SpecialAgent exposes a simple pattern for configuration of SpecialAgent, the [Integrations](#63-integration), as well as [Trace Exporters](#62-trace-exporter). The configuration pattern is based on system properties, which can be defined on the command-line, in a properties file, or in [@AgentRunner.Config][agentrunner-config] for JUnit tests: **Configuration Layers** 1. Properties passed on the command-line via `-D${PROPERTY}=...` override same-named properties defined in the subsequent layers. 1. The [@AgentRunner.Config][agentrunner-config] annotation allows one to define log level and re/transformation event logging settings. Properties defined in the `@Config` annotation override same-named properties defined in the subsequent layers. 1. The `-Dsa.config=${PROPERTIES_FILE}` command-line argument can be specified for SpecialAgent to load property names from a `${PROPERTIES_FILE}`. Properties defined in the `${PROPERTIES_FILE}` override same-named properties defined in the subsequent layer. 1. The SpecialAgent has a `default.properties` file that defines default values for properties that need to be defined. ### 3.2 Properties The following properties are supported by all [Integration Rules](#64-integration-rule): #### 3.2.1 Logging * `-Dsa.log.level` Set the logging level for SpecialAgent. Acceptable values are: `SEVERE`, `WARNING`, `INFO`, `CONFIG`, `FINE`, `FINER`, or `FINEST`, or any numerical log level value is accepted also. The default logging level is set to `WARNING`. * `-Dsa.log.events` Set the re/transformation events to be logged: `DISCOVERY`, `IGNORED`, `TRANSFORMATION`, `ERROR`, `COMPLETE`. The property accepts a comma-delimited list of event names. By default, the `ERROR` event is logged (only when run with `AgentRunner`). * `-Dsa.log.file` Set the logging output file for SpecialAgent. #### 3.2.2 Integration 1. Verbose mode `-Dsa.integration.*.verbose`
`-Dsa.integration.${INTEGRATION_NAME_PATTERN}.verbose` Sets verbose mode for all plugins (i.e. `*`) or one plugin (i.e. `${INTEGRATION_NAME_PATTERN}`). This property can also be set in an `AgentRunner` JUnit test with the `@AgentRunner.Config(verbose=true)` for all tests in a JUnit class, or `@AgentRunner.TestConfig(verbose=true)` for an individual JUnit test method. The [Java Concurrent API plugin](https://github.com/opentracing-contrib/java-concurrent) supports verbose mode, which is disabled by default. To enable, set `sa.integration.concurrent.verbose=true`. In non-verbose mode, parent span context is propagating to task execution (if a parent span context exists). In verbose mode, a parent span is always created upon task submission to the executor, and a child span is created when the task is started. #### 3.2.3 General 1. Skip fingerprint verification: `-Dsa.fingerprint.skip` Tells the SpecialAgent to skip the fingerprint verification when linking [Integrations](#63-integration) into class loaders. This option allows one to work around an unexpected fingerprint verification failure, which can happen in complex runtimes that do not contain all class definitions on the class path. It must be noted, however, that if the fingerprint verification is disabled, the SpecialAgent will indiscriminately install all plugins regardless of library version compatibility issues, which may lead to `NoClassDefFoundError`, `IllegalAccessError`, `AbstractMethodError`, `LinkageError`, etc. ### 3.3 Selecting the [Trace Exporter](#62-trace-exporter) The SpecialAgent supports OpenTracing-compatible [Tracers](#61-tracer). There are 2 ways to connect a [Tracer](#61-tracer) to the SpecialAgent runtime: 1. **Bundled [Trace Exporters](#62-trace-exporter)** The SpecialAgent bundles the following [Trace Exporters](#62-trace-exporter): 1. [Jaeger Trace Exporter](https://github.com/opentracing-contrib/java-opentracing-jaeger-bundle) 1. [LightStep Trace Exporter](https://github.com/lightstep/lightstep-tracer-java/tree/master/lightstep-tracer-jre-bundle) 1. [Wavefront Trace Exporter](https://github.com/wavefrontHQ/wavefront-opentracing-bundle-java) 1. [OpenTelemetry Bridge Trace Exporter](https://github.com/opentracing-contrib/java-opentelemetry-bridge) 1. [`MockTracer`](https://github.com/opentracing/opentracing-java/blob/master/opentracing-mock/) The `-Dsa.exporter=${SHORT_NAME}` property specifies which [Trace Exporter](#62-trace-exporter) is to be used. The value of `${SHORT_NAME}` is the [Short Name](#21221-short-name) of the [Trace Exporter](#62-trace-exporter), i.e. `jaeger`, `lightstep`, `wavefront`, `otel`, or `mock`. 1. **External [Trace Exporters](#62-trace-exporter)** The SpecialAgent allows external [Trace Exporters](#62-trace-exporter) to be attached to the runtime. The `-Dsa.exporter=${TRACE_EXPORTER_JAR}` property specifies the JAR path of the [Trace Exporter](#62-trace-exporter) to be used. The `${TRACE_EXPORTER_JAR}` must be a JAR that supplies an implementation of the [`TracerFactory`](https://github.com/opentracing-contrib/java-tracerresolver#tracer-factory) interface of the [TracerResolver](https://github.com/opentracing-contrib/java-tracerresolver) project. _**NOTE**: If a tracer is not specified with the `-Dsa.exporter=...` property, the SpecialAgent will present a warning in the log that states: `Tracer NOT RESOLVED`._ ### 3.4 Disabling [Integration Rules](#64-integration-rule) [Integrations](#63-integration) bundled with the SpecialAgent are enabled by default. Multiple properties to disable or to enable all or individual plugins can be declared via the [Configuration Pattern](#3-configuration). The processing order of the properties is equal to the order of their declaration. #### 3.4.1 Disabling All [Integration Rules](#64-integration-rule) To disable _all [Integrations](#63-integration)_: ``` sa.integration.*.disable ``` The suffix `.disable` is interchangeable with `.enable=false`. #### 3.4.2 Disabling (or enabling) One [Integration Rule](#64-integration-rule) To disable _an individual [Integration](#63-integration)_: ``` sa.integration.${INTEGRATION_NAME_PATTERN}.disable ``` The suffix `.disable` is interchangeable with `.enable=false`. Conversely, to enable _an individual [Integration](#63-integration)_. ``` sa.integration.${INTEGRATION_NAME_PATTERN}.enable ``` The suffix `.enable` is interchangeable with `.disable=false`. ##### Integration Name Pattern The value of `${INTEGRATION_NAME_PATTERN}` represents the name of the [Integration Rule](#64-integration-rule), as specified in [Integrations](#41-integrations) ("Integration Rule" column). The `${INTEGRATION_NAME_PATTERN}` allows for the use of `*` and `?` characters to match multiple rules simultaneously. For instance: 1. `dubbo:2.?`
Matches all Dubbo rules, including `dubbo:2.6`, and `dubbo:2.7`. 1. `cassandra:driver:*`
Matches all Cassandra Driver rules, including `cassandra:driver:3`, and `cassandra:driver:4`. 1. `spring:web*`
Matches all Spring WebMVC, Spring WebFlux and Spring WebSocket rules, including `spring:webmvc`, `spring:webflux`, and `spring:websocket`. 1. `cassandra:driver`
Matches all Cassandra Driver rules, including `cassandra:driver:3` and `cassandra:driver:4`. If the _version part_ of the `${INTEGRATION_NAME_PATTERN}` does not end with a `*` or `?` character, a `*` will be appended automatically. Therefore: 1. `dubbo:2`
Matches all Dubbo v2 rules, including `dubbo:2.6`, and `dubbo:2.7`. 1. `cassandra:driver`
Matches all Cassandra Driver rules, `cassandra:driver:3`, and `cassandra:driver:4`. 1. `spring`
Matches all Spring rules. 1. `spring:w`
Does not match any rules. #### 3.4.3 Disabling `AgentRule`s of an [Integration Rule](#64-integration-rule) To disable _an individual `AgentRule` of an [Integration](#63-integration)_: ``` sa.integration.${INTEGRATION_NAME_PATTERN}#${AGENT_RULE_SIMPLE_CLASS_NAME}.disable ``` The suffix `.disable` is interchangeable with `.enable=false`. The value of `${AGENT_RULE_SIMPLE_CLASS_NAME}` is the simple class name of the `AgentRule` subclass that is to be disabled. ### 3.5 Disabling [Trace Exporters](#62-trace-exporter) All [Trace Exporters](#62-trace-exporter) bundled in SpecialAgent are enabled by default, and can be disabled. To disable _all [Trace Exporters](#62-trace-exporter)_: ``` sa.exporter.*.disable ``` The suffix `.disable` is interchangeable with `.enable=false`. To disable _an individual Trace Exporter_: ``` sa.exporter.${SHORT_NAME}.disable ``` The suffix `.disable` is interchangeable with `.enable=false`. The value of `${SHORT_NAME}` is the [Short Name](#21221-short-name) of the plugin, such as `jaeger`, `lightstep`, `wavefront`, `otel`, or `mock`. ### 3.6 Including Custom [Integration Rules](#64-integration-rule) Custom [Integration Rules](#64-integration-rule) can be implemented by following the [SpecialAgent Rule API](https://github.com/opentracing-contrib/java-specialagent/tree/master/opentracing-specialagent-api). JARs containing custom [Integration Rules](#64-integration-rule) can be loaded by SpecialAgent via: ``` -Dsa.classpath=${JARs} ``` Here, `${JARs}` refers to a `File.pathSeparator`-delimited (`:` for \*NIX, `;` for Windows) string of JARs containing the custom [Integration Rules](#64-integration-rule). ### 3.7 Rewritable Tracer The [Rewritable Tracer](#37-rewritable-tracer) allows one to rewrite data in the spans created by [Integrations](#41-integrations) without having to modify the source code. The [Rewritable Tracer](#37-rewritable-tracer) is a rules engine that is configured via JSON files [that conform to a specification][rewrite]. For example: * The following JSON defines a rule for all [Integrations](#41-integrations) to drop all **tag**s in spans matching `key` literal `http.url` and `value` regex `.*secret.*`. ```json { "*": [ { "input": { "type": "tag", "key": "http.url", "value": ".*secret.*" } } ] } ``` * The following JSON defines a rule for the `jedis` [Integration](#41-integrations) to rewrite all **log**s matching `key` literal `http.method` as a **tag**. ```json { "jedis": [ { "input": { "type": "log", "key": "http.method", }, "output": { "type": "tag" } } ] } ``` For a configuration spec and other use-case examples, please refer to the [`rewrite` plugin][rewrite]. ## 4 Supported [Integrations](#63-integration) and [Trace Exporters](#62-trace-exporter) ### 4.1 [Integrations](#63-integration) Intrinsically, the SpecialAgent includes support for the instrumentation of the following 3rd-party libraries. Each row refers to an [Integration](#63-integration), the [Integration Rule](#64-integration-rule), and the minimum and maximum version tested by the build. For the development of [Integration Rules](#64-integration-rule), please refer to the [`opentracing-specialagent-api`][api] module. | Integration
(link to impl. of [Integration](#63-integration)) | Integration Rule
(link to impl. of [Integration Rule](#64-integration-rule)) | Min Version
(min supported) | Max Version
(max supported) | |:-|:-|:-:|:-:| | [Akka Actor](https://github.com/opentracing-contrib/java-akka) | [`akka:actor`][akka-actor] | 2.5.0 | LATEST | | Akka Http | [`akka:http`][akka-http] | 10.1.0 | LATEST | | [Apache Camel](https://github.com/apache/camel/tree/master/components/camel-opentracing) | [`camel`][camel] | 2.24.0 | 2.24.2 | | Apache CXF | [`cxf`][cxf] | 3.3.3 | LATEST | | Apache Dubbo | [`dubbo:2.6`][dubbo-2.6] | 2.6.2 | 2.6.7 | | | [`dubbo:2.7`][dubbo-2.7] | 2.7.1 | LATEST | | [Apache HttpClient](https://github.com/opentracing-contrib/java-apache-httpclient) | [`apache:httpclient`][apache-httpclient] | 4.4 | LATEST | | [Async Http Client](https://github.com/opentracing-contrib/java-asynchttpclient) | [`asynchttpclient`][asynchttpclient] | 2.7.0 | LATEST | | [AWS SDK](https://github.com/opentracing-contrib/java-aws-sdk) | [`aws:sdk:1`][aws-sdk-1] | 1.11.79 | LATEST | | | [`aws:sdk:2`][aws-sdk-2] | 2.1.4 | **FIXME** | | [Cassandra Driver](https://github.com/opentracing-contrib/java-cassandra-driver) | [`cassandra:driver:3`][cassandra-driver-3] | 3.0.0 | 3.7.2 | | | [`cassandra:driver:4`][cassandra-driver-4] | 4.0.0 | LATEST | | Couchbase Client | [`couchbase-client`][couchbase-client] | 2.7.3 | 2.7.13 | | Dynamic
| [`dynamic`
(configurable)][dynamic] | **\***
| **\***
| | [Elasticsearch Client
](https://github.com/opentracing-contrib/java-elasticsearch-client) | [`elasticsearch:client-transport`][elasticsearch-7-transport-client]
[`elasticsearch:client-rest`][elasticsearch-7-rest-client] | 6.4.0
6.4.0 | LATEST
6.8.7 | | [Feign](https://github.com/OpenFeign/feign-opentracing/tree/master/feign-opentracing) | [`feign`][feign] | 9.0.0 | LATEST | | Google Http Client | [`google-http-client`][google-http-client] | 1.19.0 | LATEST | | [Grizzly AsyncHttpClient](https://github.com/opentracing-contrib/java-grizzly-ahc) | [`grizzly:ahc`][grizzly-ahc] | 1.15 | LATEST | | [Grizzly HTTP Server](https://github.com/opentracing-contrib/java-grizzly-http-server) | [`grizzly:http-server`][grizzly-http-server] | 2.3.35 | LATEST | | [GRPC](https://github.com/opentracing-contrib/java-grpc) | [`grpc`][grpc] | 1.7.3 | LATEST | | [Hazelcast](https://github.com/opentracing-contrib/opentracing-hazelcast) | [`hazelcast`][hazelcast] | 3.12 | 3.12.6 | | [Java Concurrent API \[`java.util.concurrent`\]](https://github.com/opentracing-contrib/java-concurrent) | [`concurrent`][concurrent] | 1.5 | 11 | | [Java JDBC API \[`java.sql`\]][java-jdbc]
| [`jdbc`
(configurable)][jdbc] | 3.1
| 4.3
| | [Java JMS API \[`javax.jms`\]][java-jms] | [`jms`][jms] | 1.1-rev-1 | LATEST | | [Java Servlet API \[`javax.servlet`\]](https://github.com/opentracing-contrib/java-web-servlet-filter)
| [`servlet`
(configurable)][servlet] | 2.3
| 3.1
| | Jetty | | 7.6.21.v20160908 | 9.2.15.v20160210 | | Tomcat | | 7.0.65 | 9.0.27 | | Java Thread [`java.lang.Thread`] | [`thread`][thread] | 1.0 | 11 | | HttpURLConnection [`java.net.HttpURLConnection`] | [`httpurlconnection`][httpurlconnection] | 1.1 | 11 | | [JAX-RS Client](https://github.com/opentracing-contrib/java-jaxrs) | [`jax-rs`][jaxrs] | 2.0 | LATEST | | [Jedis Client](https://github.com/opentracing-contrib/java-redis-client/tree/master/opentracing-redis-jedis) | [`jedis`][jedis] | 2.7.0 | 3.3.0 | | [Kafka Client](https://github.com/opentracing-contrib/java-kafka-client) | [`kafka:client`][kafka-client] | 1.1.0 | LATEST | | [Kafka Streams](https://github.com/opentracing-contrib/java-kafka-client) | [`kafka:streams`][kafka-streams] | 1.1.0 | LATEST | | [Lettuce Client](https://github.com/opentracing-contrib/java-redis-client/tree/master/opentracing-redis-lettuce) | [`lettuce`][lettuce] | 5.0.0.RELEASE | LATEST | | [MongoDB Driver](https://github.com/opentracing-contrib/java-mongo-driver) | [`mongo:driver`][mongo-driver] | 3.9.0 | LATEST | | Mule 4 Artifact Module | [`mule:artifact-module:4`][mule-4-module-artifact] | 4.2.2 | LATEST | | Mule 4 Core | [`mule:core:4`][mule-4-core] | 4.2.2 | LATEST | | [Neo4j Driver](https://github.com/opentracing-contrib/java-neo4j-driver) | [`neo4j:driver`][neo4j-driver] | 4.0.0 | LATEST | | Netty | [`netty`][netty] | 4.1.0 | 4.1.46.Final | | [OkHttp][java-okhttp] | [`okhttp`][okhttp] | 3.5.0 | LATEST | | Play Framework | [`play`][play] | 2.6.0 | LATEST | | Play WS | [`play:ws`][play-ws] | 2.0.0 | **FIXME** | | Pulsar Client | [`pulsar:client`][pulsar-client] | 2.2.0 | **FIXME** | | Pulsar Functions
| [`pulsar-functions`
(configurable)][pulsar-functions] | 2.2.0
| **FIXME**
| | [RabbitMQ Client](https://github.com/opentracing-contrib/java-rabbitmq-client) | [`rabbitmq:client`][rabbitmq-client] | 5.0.0 | LATEST | | [Reactor](https://github.com/opentracing-contrib/java-reactor) | [`reactor`][reactor] | 3.2.3.RELEASE | LATEST | | [Redisson](https://github.com/opentracing-contrib/java-redis-client/tree/master/opentracing-redis-redisson) | [`redisson`][redisson] | 3.11.0 | 3.11.5 | | [RxJava](https://github.com/opentracing-contrib/java-rxjava) | [`rxjava:2`][rxjava-2] | 2.1.0 | LATEST | | | [`rxjava:3`][rxjava-3] | 3.0.0 | LATEST | | [Spring JMS](https://github.com/opentracing-contrib/java-jms/tree/master/opentracing-jms-spring) | [`spring:jms`][spring-jms] | 5.0.0.RELEASE | LATEST | | [Spring Kafka](https://github.com/opentracing-contrib/java-kafka-client/tree/master/opentracing-kafka-spring) | [`spring:kafka`][spring-kafka] | 2.2.0.RELEASE | LATEST | | [Spring Messaging](https://github.com/opentracing-contrib/java-spring-messaging) | [`spring:messaging`][spring-messaging] | 5.1.0.RELEASE | 5.3.0.RELEASE | | [Spring RabbitMQ](https://github.com/opentracing-contrib/java-spring-rabbitmq) | [`spring:rabbitmq`][spring-rabbitmq] | 2.0.0.RELEASE | LATEST | | [Spring WebFlux](https://github.com/opentracing-contrib/java-spring-web) | [`spring:webflux`][spring-webflux] | 5.1.0.RELEASE | LATEST | | [Spring Boot WebSocket STOMP](https://github.com/opentracing-contrib/java-spring-cloud/tree/master/instrument-starters/opentracing-spring-cloud-websocket-starter) | [`spring:websocket`][spring-websocket] | 2.1.0.RELEASE | LATEST | | [Spring \[`@Async` and `@Scheduled`\]](https://github.com/opentracing-contrib/java-spring-cloud/tree/master/instrument-starters/opentracing-spring-cloud-core) | [`spring:scheduling`][spring-scheduling] | 5.0.0.RELEASE | LATEST | | [Spring Web MVC](https://github.com/opentracing-contrib/java-spring-web) | [`spring:webmvc`][spring-webmvc] | 3.0.2.RELEASE | LATEST | | [Spymemcached](https://github.com/opentracing-contrib/java-memcached-client/tree/master/opentracing-spymemcached) | [`spymemcached`][spymemcached] | 2.11.0 | LATEST | | [Thrift](https://github.com/opentracing-contrib/java-thrift) | [`thrift`][thrift] | 0.10.0 | 0.13.0 | | [Zuul](https://github.com/opentracing-contrib/java-spring-cloud/tree/master/instrument-starters/opentracing-spring-cloud-zuul-starter) | [`zuul`][zuul] | 1.0.0 | 2.1.1 | ### 4.2 [Trace Exporters](#62-trace-exporter) Intrinsically, the SpecialAgent includes support for the following [Trace Exporters](#62-trace-exporter). A demo can be referenced [here](https://github.com/opentracing-contrib/java-specialagent-demo). | Trace Exporter
(link to impl. of trace exporter) | [Short Name](#21221-short-name)
(`-Dsa.exporter=${short_name}`) | |:-|:-| | [Jaeger Trace Exporter](https://github.com/opentracing-contrib/java-opentracing-jaeger-bundle)
[(configuration reference)](https://github.com/jaegertracing/jaeger-client-java/blob/master/jaeger-core/README.md#configuration-via-environment) | `jaeger`
| | [LightStep Trace Exporter](https://github.com/lightstep/lightstep-tracer-java/tree/master/lightstep-tracer-jre-bundle)
[(configuration reference)](https://docs.lightstep.com/docs/create-projects-for-your-environments) | `lightstep`
| | [Wavefront Trace Exporter](https://github.com/wavefrontHQ/wavefront-opentracing-bundle-java)
[(configuration reference)](https://github.com/wavefrontHQ/wavefront-opentracing-bundle-java#setup-steps) | `wavefront`
| | [OpenTelemetry Bridge Trace Exporter](https://github.com/opentracing-contrib/java-opentelemetry-bridge)
(configuration reference) | `otel`
| | [`MockTracer`](https://github.com/opentracing/opentracing-java/blob/master/opentracing-mock/)| `mock` | ### 4.3 Libraries instrumented via other Integrations The following libraries are instrumented by other [Integration Rules](#64-integration-rule). | Library
| Integration Rule
(link to impl. of [Integration Rule](#64-integration-rule)) | Min Version
(min supported) | Max Version
(max supported) | |:-|:-|:-:|:-:| | Hystrix | [`concurrent`][concurrent] | 1.5 | 11 | | JDBI
| [`jdbc`
(configurable)][jdbc] | \*
| \*
| | Ratpack | [`netty`][netty] | 1.4.0 | LATEST | | Solr Client | [`apache:httpclient`][apache-httpclient] | 4.0.0 | LATEST | | SparkJava | [`javax.servlet`][servlet] | 2.2 | LATEST | | Spring Cloud | \*
| \*
| \*
| | Spring Data
| [`jdbc`
(configurable)][jdbc] | \*
| \*
| | Spring Web | [`httpurlconnection`][httpurlconnection] | \*
| \*
| | Twilio | [`apache:httpclient`][apache-httpclient] | 0.0.1 | LATEST | ## 5 Objectives ### 5.1 Goals 1. The SpecialAgent must allow any [Integration](#63-integration) available in [opentracing-contrib][opentracing-contrib] to be automatically installable in applications that utilize a 3rd-party library for which an [Integration](#63-integration) exists. 1. The SpecialAgent must automatically install the [Integration](#63-integration) for each 3rd-party library, regardless in which class loader the 3rd-party library is loaded. 1. The SpecialAgent must not adversely affect the runtime stability of the application on which it is intended to be used. This goal applies only to the code in the SpecialAgent, and transitively applies to the code of the [Integration](#63-integration) made available in [opentracing-contrib][opentracing-contrib]. 1. The SpecialAgent must support [Static Attach](#221-static-attach) and [Dynamic Attach](#222-dynamic-attach) for applications running on JVM versions 1.7, 1.8, 9, and 11. 1. The SpecialAgent must implement a lightweight test methodology that can be easily applied to a module that implements [Integration](#63-integration) for a 3rd-party library. This test must simulate: 1. Launch the test in a process simulating the `-javaagent` vm argument that points to the SpecialAgent (in order to test auto-instrumentation functionality). 1. Elevate the test code to be executed from a custom class loader that is disconnected from the system class loader (in order to test bytecode injection into an isolated class loader that cannot resolve classes on the system classpath). 1. Allow tests to specify their own `Tracer` instances via `GlobalTracer`, or initialize a `MockTracer` if no instance is specified. The test must provide a reference to the `Tracer` instance in the test method for assertions with JUnit. 1. The SpecialAgent must provide a means by which [Integrations](#63-integration) can be configured for use on a target application. ### 5.2 Non-Goals 1. The SpecialAgent is not designed to modify application code, beyond the installation of [Integrations](#63-integration). For example, there is no facility for dynamically augmenting arbitrary code. ## 6 Definitions The following terms are used throughout this documentation. #### 6.1 [Tracer](#61-tracer) Service provider of the OpenTracing standard, providing an implementation of the `io.opentracing.Tracer` interface. Examples: * [Jaeger Tracer][jaeger] * [LightStep Tracer][lightstep] * [Wavefront Tracer][wavefront] _[Tracers](#61-tracer) **are not** coupled to the SpecialAgent._ #### 6.2 [Trace Exporter](#62-trace-exporter) A bridge providing automatic discovery of [Tracers](#61-tracer) in a runtime instrumented with the OpenTracing API. This bridge implements the `TracerFactory` interface of [TracerResolver](https://github.com/opentracing-contrib/java-tracerresolver/blob/master/opentracing-tracerresolver/), and is distributed as a single "fat JAR" that can be conveniently added to the classpath of a Java process. _[Trace Exporters](#62-trace-exporter) **are not** coupled to the SpecialAgent._ #### 6.3 [Integration](#63-integration) An OpenTracing Integration for a 3rd-party library, existing as individual repositories in [opentracing-contrib][opentracing-contrib]. Examples: * [`opentracing-contrib/java-okhttp`][java-okhttp] * [`opentracing-contrib/java-jdbc`][java-jdbc] * [`opentracing-contrib/java-jms`][java-jms] _[Integrations](#63-integration) **are not** coupled to the SpecialAgent._ #### 6.4 [Integration Rule](#64-integration-rule) A submodule of the SpecialAgent that implements the auto-instrumentation rules for [Integrations](#63-integration) via the [`opentracing-specialagent-api`][api]. See [Integrations](#2121-integrations) for a description of Uncoupled and Coupled Integrations. Examples: * [`rule/okhttp`][okhttp] * [`rule/jdbc`][jdbc] * [`rule/jms-1`][jms-1] _[Integration Rules](#64-integration-rule) **are** coupled to the SpecialAgent._ ## 7 Credits Thank you to the following contributors for developing [Integrations](#2121-integrations) and [Integration Rules](#64-integration-rule): * [Sergei Malafeev](https://github.com/malafeev) * [Jose Montoya](https://github.com/jam01) * [Przemyslaw Maciolek](https://github.com/pmaciolek) * [Jianshao Wu](https://github.com/jianshaow) * [Gregor Zeitlinger](https://github.com/zeitlinger) * [@limfriend](https://github.com/limfriend) Thank you to the following contributors for developing [Trace Exporters](#62-trace-exporter): * [Carlos Alberto Cortez](https://github.com/carlosalberto) * [Han Zhang](https://github.com/hanwavefront) Thank you to the following developers for filing issues and helping us fix them: * [Louis-Etienne](https://github.com/ledor473) * [Marcos Trejo Munguia](https://github.com/mtrejo) * [@kaushikdeb](https://github.com/kaushikdeb) * [@deepakgoenka](https://github.com/deepakgoenka) * [Prometheus](https://github.com/etsangsplk) * [Randall Theobald](https://github.com/randallt) Thank you to the following individuals for all other general contributions to the codebase: * [Daniel Rodriguez Hernandez](https://github.com/drodriguezhdez) * [qudongfang](https://github.com/qudongfang) * [Pontus Rydin](https://github.com/prydin) Finally, thanks for all of the feedback! Please share your comments [as an issue](https://github.com/opentracing-contrib/java-specialagent/issues)! ## 8 Contributing Pull requests are welcome. For major changes, please [open an issue](https://github.com/opentracing-contrib/java-specialagent/issues) first to discuss what you would like to change. Please make sure to update tests as appropriate. ## 9 License This project is licensed under the Apache 2 License - see the [LICENSE.txt](LICENSE.txt) file for details. [akka-actor]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/akka-actor [akka-http]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/akka-http [apache-httpclient]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/apache-httpclient [asynchttpclient]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/asynchttpclient [aws-sdk-1]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/aws-sdk-1 [aws-sdk-2]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/aws-sdk-2 [camel]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/camel [cassandra-driver-3]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/cassandra-driver-3 [cassandra-driver-4]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/cassandra-driver-4 [concurrent]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/concurrent [couchbase-client]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/couchbase-client [cxf]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/cxf [dubbo-2.6]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/dubbo-2.6 [dubbo-2.7]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/dubbo-2.7 [dynamic]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/dynamic [elasticsearch-7-rest-client]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/elasticsearch-7-rest-client [elasticsearch-7-transport-client]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/elasticsearch-7-transport-client [feign]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/feign [google-http-client]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/google-http-client [grizzly-ahc]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/grizzly-ahc [grizzly-http-server]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/grizzly-http-server [grpc]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/grpc [hazelcast]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/hazelcast [httpurlconnection]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/httpurlconnection [jaxrs]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/jaxrs [jdbc]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/jdbc [jedis]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/jedis [jms]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/jms [kafka-client]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/kafka-client [kafka-streams]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/kafka-streams [lettuce]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/lettuce [mongo-driver]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/mongo-driver [mule-4-core]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/mule-4-core [mule-4-module-artifact]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/mule-4-module-artifact [neo4j-driver]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/neo4j-driver [netty]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/netty [okhttp]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/okhttp [play-ws]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/play-ws [play]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/play [pulsar-client]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/pulsar-client [pulsar-functions]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/pulsar-functions [rabbitmq-client]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/rabbitmq-client [reactor]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/reactor [redisson]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/redisson [rewrite]: https://github.com/opentracing-contrib/java-specialagent/tree/master/opentracing-rewrite [rxjava-2]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/rxjava-2 [rxjava-3]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/rxjava-3 [servlet]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/servlet [spring-jms]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spring-jms [spring-kafka]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spring-kafka [spring-messaging]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spring-messaging [spring-rabbitmq]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spring-rabbitmq [spring-scheduling]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spring-scheduling [spring-webflux]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spring-webflux [spring-webmvc]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spring-webmvc [spring-websocket]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spring-websocket [spymemcached]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/spymemcached [thread]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/thread [thrift]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/thrift [zuul]: https://github.com/opentracing-contrib/java-specialagent/tree/master/rule/zuul [jaeger]: https://github.com/jaegertracing/jaeger [lightstep]: https://github.com/lightstep/lightstep-tracer-java [wavefront]: https://github.com/wavefrontHQ/wavefront-opentracing-sdk-java [agentrunner-config]: https://github.com/opentracing-contrib/java-specialagent/tree/master/opentracing-specialagent-api#51-configuring-agentrunner [api]: https://github.com/opentracing-contrib/java-specialagent/tree/master/opentracing-specialagent-api [java-jdbc]: https://github.com/opentracing-contrib/java-jdbc [java-jms]: https://github.com/opentracing-contrib/java-jms [java-okhttp]: https://github.com/opentracing-contrib/java-okhttp [opentracing-contrib]: https://github.com/opentracing-contrib/ [specialagent-pom]: https://github.com/opentracing-contrib/java-specialagent/blob/master/pom.xml [circleci]: https://circleci.com/gh/opentracing-contrib/java-specialagent/tree/master [main-release]: https://repo1.maven.org/maven2/io/opentracing/contrib/specialagent/opentracing-specialagent/1.7.4/opentracing-specialagent-1.7.4.jar [main-snapshot]: https://oss.sonatype.org/content/repositories/snapshots/io/opentracing/contrib/specialagent/opentracing-specialagent/1.7.5-SNAPSHOT