[Proposal] Feature: Keyword search capability for systems

[kgpax]

What does this PR do

Allows systems to expose keywords for traces, so that the search feature of Envy can find traces by more than just the URL of the trace.

Resolves #187

New System<T> interface member:

• Adds a new interface function to System<T>: getSearchKeywords
• Systems can implement this to return a string[] containing keywords they wish the main search to find a given trace by.
• Implements for the built-in GraphQL system, where this allows searching for traces by the GQL operation name.
• Implements for the custom viewer example CocktailDB system, where this allows searching for traces by the name of the cocktail.
• Updates customizing.md README file

New ability to inject a large volume of traces:

• A somewhat "undocumented" feature for now, but if you hold shift when clicking the "Inject mock traces" option in the debug menu, it will add the mock traces 100 times over (1,300 traces).
• The purpose of this is to inject a large number of traces to be able to verify application performance.
• I had originally tried to inject the mock traces 1000 times over, (13,000 traces), but the UI performance was unacceptable...
• ...as such I will raise a ticket to look at ways to improve the UI performance at this scale.
• For the performance of the search at 1,300 traces, the performance was in keeping with any other operation (such as filtering by system, opening a trace, etc)...
• ...and so whilst the search could still be optimised further, I don't believe that it has impacted performance dramatically; certainly at the sub-1,000 trace count.

Screenshots

Using the search term to find CocktailDb API traces by cocktail name

How to test

Basic:

• Run ENVY locally in development mode
• Inject the mock traces (or, if you're brave, inject lots of them by holding shift when clicking the option)
• Search for one of the GQL operation names, such as RegisterPerson or PDP
• You should see these results appear.
• Compare this to existing versions, where these GQL results will not appear.

Custom viewer:

• Run yarn example:apollo:custom-viewer
• Open ENVY on http://localhost:4002/
• Open the example apollo-client on http://localhost:4001/
• Refresh the cocktail widget a few times, make a note of a cocktail name you want to search for
• Search for a cocktail name
• Matching results from the www.thecocktaildb.com API should be displayed

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: 6569a0138dbbea0dc2d33ea8eda31f0301b95ad6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages

| Name | Type | | -------------- | ----- | | @envyjs/webui | Minor | | @envyjs/apollo | Minor | | @envyjs/core | Minor | | @envyjs/nextjs | Minor | | @envyjs/node | Minor | | @envyjs/web | Minor |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[carbonrobot]

Should we consider a solution that is system-agnostic and uses the object keys (unfiltered or filtered) of incoming traces with a hash map or stringly based search? That could allow users to search on any value available without specifically updating the code.

The stringly based search (where you concat fields into a single string) is less performant and higher likely hood of false positives, but simpler to implement.

The hash map (where you map all incoming keys/value pairs into a map) is higher performance, but a little more difficult to implement.

We could also consider a tag-based search approach where you can type the specific property into the search host:localhost, the same way chrome network tools work, but that could be quite a bit more work.

[kgpax]

Should we consider a solution that is system-agnostic and uses the object keys (unfiltered or filtered) of incoming traces with a hash map or stringly based search? That could allow users to search on any value available without specifically updating the code.

The stringly based search (where you concat fields into a single string) is less performant and higher likely hood of false positives, but simpler to implement.

The hash map (where you map all incoming keys/value pairs into a map) is higher performance, but a little more difficult to implement.

We could also consider a tag-based search approach where you can type the specific property into the search host:localhost, the same way chrome network tools work, but that could be quite a bit more work.

We can. That's certainly an option. Some considerations:

• You might want to the ability to refine what parts of the data should be discoverable in search. For example, searching for "Classic XXI" might bring back the trace you are looking for because that corresponds to the product name, but it might also bring back a lot of traces you didn't want, because that shoe was mentioned in the product story, or in the basket details, etc... things which are technically valid, but not the primary piece of data you wanted to search.
• You might want to transform the data which is available for search; for example, if you wanted to be able allow search for data found in cookies, but you want to decode that data rather than expect people to enter encoded values in a search.

The tag-based approach is a good idea, and one which can work alongside the "generic" search. E.g., host:localhost method:PATCH classic XXI could be a search term which looks for specifically localhost in the host data, PATCH in the method data, and classic XXI in the same way defined in this PR