Log to trace utility

[agardnerIT]

It is very common in legacy environments (ie. those not using traces) for correlation IDs to be added to log line entries.

(Human) operators can then search their logging software to identify "something" across tiers or logs.

It works, but the visualisation is highly dependent on the available visualisations provided by the logging software and thus either difficult, non-standard or hard to do.

Take these log lines for example:

2024-05-10 07:00:00 INFO Wake Up corr_id=abc123499
2024-05-10 08:00:00 INFO Breakfast corr_id=abc123499
2024-05-10 08:30:00 INFO Drive to Work corr_id=abc123499
2024-05-10 09:00:00 INFO Work corr_id=abc123499
2024-05-10 17:00:00 INFO Drive Home corr_id=abc123499
2024-05-10 18:00 INFO At Home corr_id=abc123499
2024-05-10 22:00:00 INFO Sleep corr_id=abc123499

As humans, we can readily understand that this:

1. Describes a unique individual's day (abc123499)
2. That this individual did 5 activities (Wake up, had breakfast, Drove to work, Worked, Drove home and Slept)

But wouldn't this be more logical visualised like this?

Task

• Implement a "Log to trace" utility to help DevOps / SREs / Human operators visualise their data

Assumptions

• We can assume that tracepusher is available on the PATH

Design Decisions

• This utility should be able to consume a log file (an input is MANDATORY)
• Users of this utility should be able to specify a string to search across the log lines (eg. corr_id=abc123499) (this parameter is MANDATORY)
• Users should be able to provide a time length that the last log line item is assumed to be (eg. in the above example, we can't know how long Sleep should "last" - so the user can specify eg. 30s, 5m or 6h)

[agardnerIT]

First version of this: Tracking GitHub codespace creation using tracepusher (sample code)