Support Tail Based Sampling Processor From OTEL Collector Extension

mahadzaryab1 commented 2 months ago

Which problem is this PR solving?

Closes #5867

Description of the changes

Added the tail-based sampling processor extension from otel to jaeger
Added a docker compose to demonstrate usage of the tail-based sampling processor extension in jaeger.
Added an end to end integration test to test that the new processor works as expected
Added a README to the docker compose setup describing the setup and usage of the new processor

How was this change tested?

An end to end integration test was added and is run from the CI

Checklist

[x] I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
[x] I have signed all commits
[x] I have added unit tests for the new functionality
[x] I have run lint and test steps successfully
- for jaeger: make lint test
- for jaeger-ui: yarn lint and yarn test

codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 96.82%. Comparing base (9a30dfc) to head (c50ded3). Report is 1 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #5878 +/- ## ========================================== - Coverage 96.83% 96.82% -0.02% ========================================== Files 342 342 Lines 16524 16525 +1 ========================================== - Hits 16001 16000 -1 - Misses 337 339 +2 Partials 186 186 ``` | [Flag](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | Coverage Δ | | |---|---|---| | [badger_v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `8.05% <ø> (ø)` | | | [badger_v2](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `1.82% <ø> (ø)` | | | [cassandra-3.x-v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `16.62% <ø> (ø)` | | | [cassandra-3.x-v2](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `1.75% <ø> (ø)` | | | [cassandra-4.x-v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `16.62% <ø> (ø)` | | | [cassandra-4.x-v2](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `1.75% <ø> (ø)` | | | [elasticsearch-6.x-v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `18.79% <ø> (+0.01%)` | :arrow_up: | | [elasticsearch-7.x-v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `18.85% <ø> (+0.01%)` | :arrow_up: | | [elasticsearch-8.x-v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `19.03% <ø> (ø)` | | | [elasticsearch-8.x-v2](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `1.82% <ø> (ø)` | | | [grpc_v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `9.49% <ø> (ø)` | | | [grpc_v2](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `7.16% <ø> (ø)` | | | [kafka-v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `9.74% <ø> (ø)` | | | [kafka-v2](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `1.82% <ø> (ø)` | | | [memory_v2](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `1.82% <ø> (ø)` | | | [opensearch-1.x-v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `18.90% <ø> (+0.01%)` | :arrow_up: | | [opensearch-2.x-v1](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `18.90% <ø> (ø)` | | | [opensearch-2.x-v2](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `1.82% <ø> (ø)` | | | [tailsampling-processor](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `0.46% <ø> (?)` | | | [unittests](https://app.codecov.io/gh/jaegertracing/jaeger/pull/5878/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing) | `95.30% <100.00%> (-0.02%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=jaegertracing#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

yurishkuro commented 2 months ago

I tested the binary sizes

without this change - 75Mb
with this change - 114Mb
with only tail sampler (without load balancer) - 78Mb

I would suggest we only include tail sampler. That's the component that's most useful in the final collector, which needs to be Jaeger. But the upstream load balancing collectors can be just OTEL Collectors.

yurishkuro commented 2 months ago

make sure to do git pull, I pushed updates to go.mod to resolve conflicts

mahadzaryab1 commented 2 months ago

make sure to do git pull, I pushed updates to go.mod to resolve conflicts

Thank you so much for doing this for me!

mahadzaryab1 commented 2 months ago

@yurishkuro i'm really sorry i made a mistake in rebasing off main - would you be able to fix the conflicts again?

mahadzaryab1 commented 2 months ago

@yurishkuro the README and the rest of the PR is ready for review now

mahadzaryab1 commented 2 months ago

@yurishkuro - there looks to be a failing test in the CI. Is it a flaky test? I don't believe its related to my changes.

yurishkuro commented 2 months ago

🎉 🎉 🎉

jaegertracing / jaeger