tailview cannot detect my log formmat

[shokri-navid]

I have a log file which is written with this format (serilog format) : "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} <source:{SourceContext}>{NewLine}{Exception}" those are like this:

2020-09-13 00:00:12.207 +04:30 [INF] Fetch modification job triggered at 9/13/2020 12:00:12 AM!!! 2020-09-13 00:00:13.046 +04:30 [INF] 1 modified account received for date 1399/06/23
2020-09-13 00:00:28.524 +04:30 [INF] Personage modification job triggered at 9/13/2020 12:00:28 AM!!! 2020-09-13 00:00:28.527 +04:30 [INF] 1 items are in queue to process 2020-09-13 00:00:34.402 +04:30 [INF] User with NationalCode YXZ-OPRRTTT-1 was not in system and account hasbeen created with personage Id 603860 2020-09-13 00:00:37.300 +04:30 [INF] User created for Personage Id 603860 and sms has been sent to 0910XXXXXXX 2020-09-13 00:00:38.398 +04:30 [INF] Customer Modified in personage service with NationalCode YXZ-OPRRTTT-1 and PersonageId 603860 2020-09-13 00:01:08.396 +04:30 [INF] Personage modification job triggered at 9/13/2020 12:01:08 AM!!! 2020-09-13 00:01:08.399 +04:30 [INF] 0 items are in queue to process 2020-09-13 00:01:38.402 +04:30 [INF] Personage modification job triggered at 9/13/2020 12:01:38 AM!!! 2020-09-13 00:01:38.405 +04:30 [INF] 0 items are in queue to process

tailview cannot detect any log level of any entry itself and also there is no way to define my log format to colorize and columnize it by tailview.

this is my tailview output view:

[Kittyfisto]

@navid-dada At the moment, Tailviewer looks for a few hardcoded keywords to guess the log level of a column, [INF] unfortunately happens to not be one of them.

In the long run, I've planned to allow users to configure which keywords to look for, but this hasn't even begun.

In the short run, I'll familiarize myself with the serilog text output format and see if I can add support for it. It doesn't look particularly complicated so you can expect to hear back from me with a solution within a week.

Hope this helps :)

[shokri-navid]

sounds great. If you think I can help you with this feature please let me know and guide me about contribution rules and steps.

[Kittyfisto]

Help is always welcome. I do have a light contribution guide, but unfortunately no particular guide to really help you start developing for Tailviewer (yet).

At the moment I'm developing a proof of concept to see if my idea of solving this actually works. I envision the following:

• A user should be able to add specific log file formats for specific logging frameworks (such as Serilog) through the UI
• Example: A user clicks on "Add new format", specifies the type "Serilog" and then enters the serilog format string, {Timestamp:dd/MM/yyyy HH:mm:ss K} [{Level}] {Message}
• Tailviewer matches the content of the log file to one of the available formats, if none is found, the default (the current behavior) is used
• The format allows Tailviewer to interpret timestamp values (necessary when merging multiple files) as well as levels (which is what your issue is about)
• In the future, this will allow a tabular display of log files, however this is even further away than what I've outlined here, unfortunately :)

If this works as I envision it (and matching can occur relatively fast enough), then Tailviewer should be able to automatically figure out which type of log file was opened and the only thing a user needed to provide was the format string used.

In the end, I value all types of contributions, be it in the form of critique of concept, implementation, testing, writing code, etc...´ Would you want to help with any of these?

[Kittyfisto]

@navid-dada It took a bit longer than expected, but I've now reached a point where Tailviewer is able to parse and interpret custom serilog files.

The workflow is as follows:

1. The user has to add a new custom log file format to tailviewer. This is done through the settings page: 1.1 Click the plus icon to add new custom log file format 1.2 Fill out the form: Name, Encoding (in case auto detect gets it wrong) and the format string. For example, {Timestamp:yyyy-MM-dd HH:mm:ss.fff K} [{Level:u3}] {Message} for the log file from your initial report
2. Open a log file. Tailviewer will, upon opening a log file for the first time, attempt to match the content to a known format. Since the user has registered such a format in step 1, Tailviewer will be able to match it and use its custom parser to interpret the contents of the log file

Configuration:

Log file display: (I've taken the liberty and changed the log levels from your example to show that Tailviewer is now able to detect the log level).

With current implementation, Tailviewer is able to not only interpret the level but also the timestamp of serilog files, allowing your to merge them together (via drag'n drop in the data sources panel): Tailviewer will automatically sort the entries by timestamp in ascending order.

I still have some cleaning up to do, fix up the UI and ensure that you don't have to restart Tailviewer upon changing those custom formats in the settings dialog (you may have to at this time), but in the meantime feel free to try out this build:

https://ci.appveyor.com/project/Kittyfisto/tailviewer/builds/35372919/artifacts

I will post further updates once cleanup is complete and when this feature is finally released.

edit I forgot to add the new plugin to the installer and had to replace the initial link with the one which includes everything in the installer to reproduce the result from the screenshots above.

[Kittyfisto]

Available with v0.9.3.889.