Ignore Non-JSON Prefix of Log Lines

ebekebe commented 2 years ago

I regularly work with tools like docker-compose or skaffold, which combine the logs of multiple processes/services and prefix them with the service name. This prefixing prevents pino-pretty from parsing the JSON in the line and replacing it with a formatted string. Additionally, log lines are prefixed differently in error cases.

I would like to propose a mode, where pino-pretty detects JSON even if it is prefixed with non-json strings.

My work arounds so far include stripping off the prefix with sed by removing everything before the first {. This worked everywhere so far, as the prefixes never included {. So implementing this kind of simple matching would already solve all my usability issues.

A more stable implementation could look if the last character in the line is a } and start parsing from the back.

Example:

Log output

[service1] {"msg": "from service1"}
[service2] {"msg": "from service2"}
error in service2: {"msg": "BOOOM"}

Transformed log output

[service1] from service1
[service2] from service2
error in service2: BOOM

jsumners commented 2 years ago

My opinion is that it would be a very difficult feature to maintain and goes against the "do one thing well" principle. This tool's sole purpose is to parse and reformat NDJSON.

I recommend using existing tooling to remove the offending prefixes prior to feeding it into pino-pretty. In the case of docker-compose:

❯ docker-compose --help logs

Usage:  docker compose logs [OPTIONS] [SERVICE...]

View output from containers

Options:
  -f, --follow          Follow log output.
      --no-color        Produce monochrome output.
      --no-log-prefix   Don't print prefix in logs.
      --since string    Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)
      --tail string     Number of lines to show from the end of the logs for each container. (default "all")
  -t, --timestamps      Show timestamps.
      --until string    Show logs before a timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)

The --no-log-prefix option would solve the problem.

mcollina commented 2 years ago

I don't think this is doable in a reliable way.

pinojs / pino-pretty

Ignore Non-JSON Prefix of Log Lines #384