Introduction

Bithon is a word combining binocular together with python.

It targets application metrics, logging, distributed tracing, alert and application risk governance under microservice environment.

Architecture

The above pic illustrates the main components of this project, including:

• Agent, which collects metrics/tracing logs automatically from client application without any code modification at the application side
• Collector, which provides various interfaces (including OpenTelemetry GRPC interface) to receive metrics/tracing logs from clients
• Pipeline, which provides a flexible and robust way to hande small data scale to a very huge data scale for incoming metrics or tracing logs
• Storage, which provides an abstraction to underlying storages like H2, MySQL or Clickhouse
• Alerting, which allows us to set up alerts by using MetricSQL style expression on existing metrics or tracing logs
• Web, which provides a simple web portal for metrics/tracing log visualization

Highlights

• Around 200 built-in metrics for various JDK or various Java middlewares like Apache Http Components
• Open Telemetry Tracing standard support and integration
• Built-in debugging diagnosis commands for target application
• Flexible deployment to adapt small data scale and huge data scale use cases
• Fast queries and very low storage cost benefit from ClickHouse
• PromQL style alerting expression support

Reference:

• White Paper
• How does the agent work?
• What's the difference between Jaeger and Bithon?
• What's the difference between OpenTelemetry and Bithon?

Demo

A demo is provided by this demo repo with a docker-compose file. You can follow the README on that demo repo to start the demo within just 3 steps.

Build

1. clone source code

After cloning this project, remember to clone the submodules by following command

git submodule update --init

2. choose a right JDK

Since the project is built upon SpringBoot 3.0, a JDK 17 or higher is required to build this project. If you have multiple JDKs on your machine, use export JAVA_HOME={YOUR_JDK_HOME} command to set correct JDK.

For example

export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-17.jdk/Contents/Home

3. build the project

For the first time to build this project, use the following command to build dependencies first:

mvn clean install --activate-profiles shaded,jooq

and then execute the following command to build the project.

mvn clean install -DskipTests

After the first build, we don't need to build the dependencies anymore unless there are changes in these dependencies.

Run

Once the project has been built, you could run the project in a standalone mode to evaluate this project.

1. Launch the server all in one

To launch server in evaluation mode, execute the following command:

java -Dspring.profiles.active=all-in-one -jar server/server-starter/target/server-starter.jar

By default, the application opens and listens on following ports at local

Once the application has started, visit http://localhost:9897/web/home to view the monitor.

Note: -Dspring.profiles.include parameter here is just for demo.

You can make changes to server/server-starter/src/main/resources/application.yml to reflect your own settings.

You can also use enable Alibaba Nacos as your configuration storage center.

2. Attach agent to your java application

Attach agent to your java agent by adding the following VM arguments.

-javaagent:<YOUR_PROJECT_DIRECTORY>/agent/agent-distribution/target/agent-distribution/agent-main.jar -Dbithon.application.name=<YOUR_APPLICATION_NAME> -Dbithon.application.env=<YOUR_APPLICATION_ENV>

By default, the agent connects collector running at local(127.0.0.1). Collector address could be changed in file agent/agent-main/src/main/resources/agent.yml. Make sure to re-build the project after changing the configuration file above.

JDKs Compatibility

Even the project is built by JDK 17 and above, the agent is compatible with JDK 1.8+. The following matrix lists the JDKs that are compatible with the agent on macOS. And in theory, this matrix works both for Windows and Linux.

JDK 11 and above

If the target application runs under JDK 11 and above, the following arguments should be added to JVM command to allow the agent to use Java Reflection on corresponding packages.

--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.net.www=ALL-UNNAMED

Supported Components

Restrictions

1. For Apache Druid, the Jersey plugin is required to be enabled to collect query information.
2. For Apache Kafka clients 3.7, the consumer metrics only works when the group.protocol is configured as classic which is the default configuration of the consumer client.
3. For Lettuce, the tracing support is only available when it's used with Spring Data Redis API.

User Doc

1. Metrics
2. Tracing
3. Logging
4. Diagnosis
5. Configuration
6. SDK
   1. Metrics
   2. Tracing

Contribution

To develop for this project, intellij is recommended.

A code style template file(dev/bithon_intellij_code_style) must be imported into intellij for coding.

For more information, check the development doc.

License

Apache License, Version 2.0