Support for custom logstash @fields

[reezer]

In many situations it would be really helpful to support custom @fields properties. Maybe in a similar way to tags. This would allow adding more structure to logging of events.

[nlf]

That's a great idea. I'll have to think about how to expose it in a way that won't break the other transports.. I'm open to suggestions if you have any.

[kopertop]

How about allowing the logging functions to take an object of tags?

[reezer]

In that case I wouldn't call it tags. But something in this direction would make sense. Maybe a new option handled similar to tags in other transport would make sense.

[nlf]

Adding something like a .meta() option would allow for this, but raises complications with the other transports. It's hard to implement something like this without causing the whole thing to become too heavy and complicated, which is exactly what I was trying to avoid.

I'll do some more thinking on this and see if I can come up with another solution. I feel like this codebase is in need of some love anyway, so maybe I can kill a few birds all at once.

[d-oliveros]

+1

[matteofigus]

+1 - perhaps I can help with this?

[kopertop]

That would be awesome. I haven't had any time to get started on this, so have at it!

[nlf]

I'm having a hard time deciding if this should be its own thing, or if tags should just optionally be an object.. If you specify tags as an array of strings, behavior is the same as now. If you specify it as an object, we use it for both tags (using Object.keys) and provide the intact object to the transports so that they can deal with it as they like. Does that make sense to everyone?

[reezer]

I am worried about this approach, as it may result in .tags({tags: ['example']}), which can be confusing.

Wouldn't it make sense to have something like .fields(object) (or custom(object), should it be able to overwrite tags) and give it to the service to handle it (for example it could use all the values, stringify and attach it, concatenate it with tags or similar things.

A completely different approach would be making things service dependent, throwing an error, if some functionality isn't available.

[nlf]

I'm more worried about that approach, as now you're adding functionality specific to one transport. The idea behind bucker was to keep it as simple as possible, not to try to cover every single feature each of its various transports can use.

Something that could work is rather an applying util.format to the body of the log message in the core of bucker, I simply pass the full message to the transport for them to deal with.

That means that, for example

logger.tags(['debug', 'request']).info('got a request', req.headers);

Would yield a payload to the transport something like

function log(time, level, module, data, tags) {
  /*
    level = 'info'
    tags = ['debug', 'request']
    data = ['got a request', { 'content-type': 'application/json', 'content-length': 498 }]
  */
}

This way what to do with the parameters passed in becomes up to the transport, which means that the logstash transport could apply objects to @fields without issue, but other transports could simplify stringify the object.

Does that work for everyone?

[kopertop]

That sounds like the best approach to me.

---

Sent from my iPhone Chris Moyer

On Mar 12, 2014, at 1:16 PM, Nathan LaFreniere notifications@github.com wrote:

I'm more worried about that approach, as now you're adding functionality specific to one transport. The idea behind bucker was to keep it as simple as possible, not to try to cover every single feature each of its various transports can use.

Something that could work is rather an applying util.format to the body of the log message in the core of bucker, I simply pass the full message to the transport for them to deal with.

That means that, for example

logger.tags(['debug', 'request']).info('got a request', req.headers); Would yield a payload to the transport something like

function log(time, level, module, data, tags) { / level = 'info' tags = ['debug', 'request'] data = ['got a request', { 'content-type': 'application/json', 'content-length': 498 }] / } This way what to do with the parameters passed in becomes up to the transport, which means that the logstash transport could apply objects to @fields without issue, but other transports could simplify stringify the object.

Does that work for everyone?

— Reply to this email directly or view it on GitHub.

[thelinuxlich]

+1

[korpa]

Any news on this?

[nlf]

Sorry, haven't had a chance to work on it yet. I'll try to get to it today.

[nlf]

This is available in the 2.0.0 alphas currently

[korpa]

Hi @nlf,

Could you please provide an example?

Thanks