search

jaegertracing / jaeger

CNCF Jaeger, a Distributed Tracing Platform
https://www.jaegertracing.io/
Apache License 2.0
20.22k stars 2.41k forks source link

Allow wildcard service search for elasticsearch storage #1148

Open ryansos opened 5 years ago

ryansos commented 5 years ago

Requirement - what kind of business use case are you trying to solve?

We have a need to collate traces across different services that share a specific tag value. The traces are initiated by separate HTTP requests and therefore appear as separate traces under separate services. We would like to search by a tag across all services with all matches displaying regardless of service name.

Problem - what in Jaeger blocks you from solving the requirement?

The jaeger-ui and the jaeger-query modules require service as a parameter.

Proposal - what do you suggest to solve the problem or improve the existing situation?

The jaeger-ui should include an 'All' or 'Any' option in the services dropdown.

image

The jaeger-query module should handle a request containing the value service=all or service=any by creating a new wildcard query. The elastic.NewWildcardQuery is supported by the bool.Must query builder.

Any open questions to address

I have a working example with tests but would like to discuss prior to opening a pull request.

I understand that Cassandra (the default storage) does not inherently support wildcard queries. How would you suggest to programmatically allow this only if the storage is elasticsearch? The query_parser module requires some changes that would potentially break Cassandra implementations.

pavolloffay commented 5 years ago

How would you suggest to programmatically allow this only if the storage is elasticsearch.

Storage implementation supporting all/any could return it from GetServices but perhaps c* could also support such query by leaving out service_name in queryByTag.

yurishkuro commented 5 years ago

afaik Zipkin supports this for Cassandra by iterating through all service names and executing queryByTag. It's expensive but doable, provided that the service graph is small enough (e.g. for Uber it would have to iterate through 3000 names, I would probably want to disable such functionality).

ryansos commented 5 years ago

@pavolloffay I will investigate this option. I'll likely also need to have some type of front-end configuration option that will toggle this feature depending on storage type.

@yurishkuro I will explore this by potentially setting a configuration value to allow/disallow wildcard searching. If Cassandra does support this, as you say, this could alleviate my issue with developing this feature as storage-specific.

rleiwang commented 3 years ago

I had just implemented an open source storage backend for Jaeger that supports wild card search, https://github.com/chronowave/opentelemetry. The current implementation supports single node only though, does it help in your use cases?