[Logs UI] Add rule based colors to log message

[weltenwort]

In order to support the visual pattern recognition performed by human vision, it would be helpful to colorize the log message according to rules.

Rules could be...

• built-in for known message types (e.g. log level for syslog messages)
• adaptable via the UI by the user
• based on literal value matching of a specific field value
• based on regex matching of a specific field value
• based on thresholds of a specific field value

The colors could...

• apply to the whole line
• just to segments of the line (e.g. the segment that encodes the source of the line)
• apply to a marker at a well-defined position in the line (e.g. a signs column at the beginning as found in many editors and IDEs)
• be continuous based on numeric field values
• categories based on known field values

[elasticmachine]

Pinging @elastic/infrastructure-ui

[makwarth]

++. For unknown messages, I'd have a simple base rule to color all field:value occurrences. Values can get a color depending on determined type.

[kanadaj]

Alternatively you could place the color on a smaller marker at the beginning of the line (like a colored side border or shape) or only on the timestamp.

[weltenwort]

Good idea, I have added symbol markers to the description.

[lssilva]

That would be a great feature, it seems it was first suggested in 2013 or earlier.

[mukeshelastic]

@katrin-freihofner and I discussed this further and in first iteration, we decided to scope it down to simpler, pre-defined rules for known keywords. We will start highlighting log.level indicators { ERROR, WARN, DEBUG } with predefined colors to make it easy for users to spot these messages without needing to use highlight text box. So here is the proposed UX:

1. Only highlight log lines that we know are errors, warns or debug messages (log.level is one indicator)
2. Only highlight the word that represents the type of message - that is only highlight error, warn, debug words.
3. If users are already highlighting any one of {error, warn, debug}, then the highlight color overrides the colors for these keywords.

@katrin-freihofner feel free to chime in if I misrepresented or missed something.

[kanadaj]

@mukeshelastic It'd be nice to make the field and the list of matched keywords for each level configurable at the very least in the beginning. Different logging systems may have different wording for the same level of errors, e.g. [INF, info, information] or [Critical, Fatal, FTL, CTC, CRT]

[mukeshelastic]

Agree with the use case @kanadaj. If we are accounting for errors, warn statements in logs then ideally we should cover keywords that are typically used for errors and warns in different logging systems. And instead of hard-coding that in our UI, we could provide flexibility to users to specify the keywords and associated colors. We will ideate on UX to enable this further.

[katrin-freihofner]

As I don't see this on our short-term roadmap I'm going to close this issue.

[weltenwort]

@katrin-freihofner are we closing out all issues not on the short-term roadmap? Where do we want to keep track the enhancement requests and ideas instead?

[katrin-freihofner]

Let's reopen so we can still keep track but remove the design label!?

[weltenwort]

That sounds reasonable, thank you!

[wolframhaussig]

I would also like to see this feature in Kibana as we were using LogTrail plugin as long as Kibana supported legacy plugins w hich had this feature. We would be using this in a logging usecase so we would be highlighting the lines based on their log level. As long as I am able to configure the loglevels in the highlighting as @kanadaj mentioned(both amount of levels and names as log4j2 supports custom loglevels) I would be happy to give it a try as soon as there is a draft. Also, I would like this feature to be integrated with #96172 so the highlighting is restored when loading a view.

[vitich]

It would be great!

[glutamatt]

if it can help there is a super hacky script for ephemeral needs to color rows backgrounds in "discover" based on column value

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)