Jaeger v2 based on OpenTelemetry collector

[yurishkuro]

Proposal: https://docs.google.com/document/d/1s4_6VgAS7qAVp6iEm5KYvpiGw3h2Ja5T5HpKo29iv00/view

This replaces the previous attempt: #3500.

Work tracked in the project https://github.com/orgs/jaegertracing/projects/3

Roadmap

Proof of concept

• [x] A single binary with memory storage and all-in-one functionality
  • [x] #4766
• [x] Validate that the current storage configuration works for primary / archive storage.
• [x] Default configuration solution (do not require config file for all-in-one)
  • [x] #4842
• [ ] Demonstrate Kafka exporter / receiver (ideally from OTEL), compatible with existing Jaeger Kafka writer
  • [ ] #4868
• [ ] Define v2 storage API that operates on OTLP batches, see if internal & remote APIs can be identical
  • [x] #5052
  • [ ] #5079
• [ ] Benchmarks #4869

Feature parity

• [ ] Extend configuration of jaeger_storage_exporter to support all existing backends -- project in https://github.com/jaegertracing/jaeger/issues/5084
• [ ] Metrics & Traces -- project in https://github.com/jaegertracing/jaeger/issues/5084
• [ ] What to do with admin port / services
• [ ] Adaptive sampling processor -- project in https://github.com/jaegertracing/jaeger/issues/5084

Prepare for Beta

• [ ] Add to integration tests
• [ ] Add to release pipeline
• [ ] Documentation

Prepare for GA

• [ ] Helm chart https://github.com/jaegertracing/helm-charts/issues/534
• [ ] Operator https://github.com/jaegertracing/jaeger-operator/issues/2411
• [ ] Clarify https://github.com/jaegertracing/jaeger#version-compatibility-guarantees

Enhancements

• [ ] SPM built-in
• [ ] Integrate with OTEL collector builder ocb
• [ ] Print active configuration (unresolved: Print final configuration OTEL #1791), https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/27003

[pavolloffay]

The V2 is as well affected by https://github.com/open-telemetry/opentelemetry-collector/pull/8768, It can cause startup failure.

[yurishkuro]

The new jaeger binary is available as Docker image https://hub.docker.com/repository/docker/jaegertracing/jaeger/tags

[yurishkuro]

I wrote down some thoughts on what Jaeger Storage API v2 might look like. Interested in feedback.

[hdost]

I wrote down some thoughts on what Jaeger Storage API v2 might look like. Interested in feedback.

So one thing that I think might be worth thinking about as well is search complexity. One issue I know we had was making search efficient on large span storage. Time range to limit is useful, but painful for users. Being able to search on attributes is a must. It might be good add criteria to compare about query patterns as well.

[88vishnu]

Appx by when Jaeger-v2 will be available with tail-based-sampling??

[yurishkuro]

I am hoping to make good progress on v2 in the Spring if we get 3 interns, and then in the summer to get it to GA.

[yasharne]

Is this open to contributing? if yes, where can I find the open tasks/issues?

[yurishkuro]

@yasharne it is definitely open, but I'd say most open issues here are medium complexity, not good-first-issue. The open issues are listed in this ticket (and more in Google docs).

[tmeckel]

When I try to use the OLTPLogExporter with Jeager and GRPC,

details collased

```python get_logger_provider().add_log_record_processor( BatchLogRecordProcessor( exporter=OTLPLogExporter( endpoint="http://jaeger:4317", insecure=True, ), ) ) ``` I recieve the following error: ``` 2024-02-27 13:41:34,233 ERROR [opentelemetry.exporter.otlp.proto.grpc.exporter] [exporter.py:306] - Failed to export logs to jaeger:4317, error code: StatusCode.UNIMPLEMENTED ``` An indication that https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/collector/logs/v1/logs_service.proto is not implemented by Jaeger. Is this correct? Or did I configured Jaeger incorrectly, ref. to following Docker Compose snippet (ports 4317 and 4318 are used only internally using the network `ipam`)? ```yaml --- # This is a Docker Compose override file to add jeager tracing version: "3.9" services: jaeger: image: jaegertracing/all-in-one:${JAEGER_VERSION:-latest} ports: - "16686:16686" environment: - LOG_LEVEL=debug - COLLECTOR_OTLP_ENABLED=true networks: - ipam network-managemnent: depends_on: - jaeger ``` https://github.com/open-telemetry/opentelemetry-collector/issues/6363

[yurishkuro]

@tmeckel Jaeger is not a logging backend, it only accepts traces.