Better trace page search

nziebart commented 6 years ago

Requirement - what kind of business use case are you trying to solve?

We would like the ability to search for spans by tag in the trace view page.

A specific use case here is finding which span is making a certain SQL query. We already know the query, but if there are hundreds of spans in the trace, it's very difficult to locate the corresponding span.

Problem - what in Jaeger blocks you from solving the requirement?

The search bar on the trace view page only searches by service and operation name (https://github.com/jaegertracing/jaeger-ui/blob/master/packages/jaeger-ui/src/selectors/span.js#L65).

Proposal - what do you suggest to solve the problem or improve the existing situation?

It would be relatively simple to also search tags and logs, by adding them to the search string. However I came across two issues when experimenting with this: 1) Some tags/logs are too big and match too many inputs. E.g. a stacktrace 2) The fuzzy search exacerbates (1). Also, some of our tag values are hash strings (e.g. 0ca9bd1a2bfcdd3d7abdff52cb49a072) which the fuzzy search loves to match against.

I'm not sure about the best solution for (1) - maybe have a way to configure tag keys to omit from search? Alternatively, require tag search to be of the form key: value (same as the homepage search feature) rather then just blindly searching all tag values against the input string.

To solve (2) I would propose either (a) removing fuzzy search (IMO it's not that useful, and users who are used to chrome search are used to needing exact matching) or (b) using fuzzy search only on operation/service name and doing exact matching for tags/logs.

Any open questions to address

Questions are noted in proposal.

bboreham commented 6 years ago

Came here looking for a way to find a span by tag value.

tiffon commented 6 years ago

@nziebart Thanks for creating this ticket!

Great suggestions and discussion of issues / solutions.

The fuzzy search exacerbates..

I agree, fuzzy search is a bit dubious... Seems fine to change to something better.

Some tags/logs are too big and match too many inputs.

The pattern used in GMail's email search seems like it would work pretty well. It starts as a simple text input which works most of the time but can be used as a more advanced search when the "▾" is clicked.

One approach would be something like the following:

Searching via just the top text input (not the "advanced" search form), would match the input string via exact substring matching against the service name, operation, or the key or value from any key:value in Tags, Process or a log message. That will probably work for most scenarios and is pretty simple.

In the form:

The (?) would be help icons which show help info when clicked
Match anything would be the same as the top text input search but with the option to use regular expressions instead of exact substring matching
Service and Operation match the service name and operation, respectively
key:value would be a logfmt based key-value search against the Tags, Process and log messages for spans
- The AND ▾ would allow the user to toggle between finding spans that match all (AND) of the key:value described or at least one (OR)
Regular expression would allow all the entered text to be processed as regular expressions. For the key:value conditions, the key and value would be separate regular expressions
Match entire key:value would indicate the key:value matching must match the entire key and entire value.
- For instance, with regular expressions enabled, the search input sql(.query)?="hello there" would match the keys sql and sql.query but not mysql or sql.params

Some benefits of the two-mode approach are:

Most use-cases are satisfied with the single text input which has a low UI foot-print and is intuitive (same as web browsers)
More sophisticated use-cases are satisfied with an easy to understand UI. It's not concise, but it's familiar and there is nothing to discover (aside from logfmt)

What do you think?

I was originally hoping to keep this much simpler, but conjunctions and differentiating between free-form text matching and key:value pair matching kept getting in my way. I wasn't able to come up with an approach using just a single text input that had an obvious UX / syntax.

nziebart commented 6 years ago

Hey @tiffon , great proposal! Although I was trying to get around the need for a more complex UI, I think it's probably the best general solution here. We've been running with a forked version of the UI that does only exact matching, but skips tags or logs that are stacktraces. This has gotten us pretty far but I could see more advanced use cases where the other options would be useful.

Instead of a big modal view like the one in the screenshot, I might suggest keeping a single search bar but having a dropdown or segmented control to the side where you can pick the "type" of search performed on the input text.

One common use case we have is searching for the existance of a certain tag. In this case it'd be nice if the key:value search mode would do partial matches. e.g. if I am looking for the tag foo.bar=baz, it should be highlighted as soon as I type f.

tiffon commented 6 years ago

@nziebart, thanks for sharing this info.

The screenshot above is intended to be a dropdown, not a modal, albeit a dropdown that's a pretty involved form. The Gmail advanced input form is shown expanded, below.

Something new I noticed is when the search is executed the parameters are converted to plain text. (And, vice-versa – the plain text is converted to the relevant form values when it's expanded.)