[feature request] Add JSON to every input line

[boenrobot]

I'd like to keep my logs both in files and in ElasticSearch, and pipe them to ES via pino-elasticsearch.

In order to keep my log files smaller, I'd like to not include data that is the same on every line, such as the application the file belongs to and perhaps other data that would be known at or before startup of pino-elasticsearch.

It would be great if I could pass a file with such a JSON from the command line or pass one directly when using as a library.

[mcollina]

I'm not understanding what you need. Can you use tee?

[boenrobot]

I plan on having the following...

A single ElasticSearch server that will store logs of multiple applications.

Each application outputs logs via pino into its own log file.

Use logagent to pipe each log file to an instance of pino-elasticsearch.

Each pino-elasticsearch instance connects to the one ElasticSearch server.

In this setup, pino-elasticsearch can't tell ElasticSearch which application the logs came in from, despite the fact that logagent does know it.

I could write the application name in the actual text of each line, but that's just wasteful.

[mcollina]

If you place all the logs in the same ES index, you’ll have to add the application name in each log line.

Why can’t you just use logagent directly?

[boenrobot]

I'm a bit new to both ElasticSearch and Logagent, so... reading up a little more on it now, it seems I could set up a different index for each app... I guess that would work.

And also reading up on logagent's parser, it supports "Bunyan format" logs which AFAIK is what pino's format is based on, so I guess that might work. The mere fact there was only "pattern" rather than an explicit plugin made me question whether it can handle it, so I thought I should pipe it to a specialized binary, and pino-elasticsearch seemed like the perfect fit.