search

winstonjs / winston-loggly

A Loggly transport for winston
http://github.com/indexzero/winston-loggly
MIT License
85 stars 110 forks source link

`isBulk` option not used #45

Open eschwartz opened 8 years ago

eschwartz commented 8 years ago

I'm trying to figure out what exactly isBulk option does.

I see that it's passed on to loggly.createClient, but loggly does not use the isBulk option..

I did notice, however, that the loggly client accepts an array of log messages, and then sends them to the bulk endpoint from the Loggly api. Would it be possible to buffer winston message, and then pass them as an array to the loggly client?

I'm asking this because I saw a big hit to performance when using winston-loggly in production, and I'm guessing because it's sending each message individually (including 6,000 - 8,000 access logs per minute, and some multiple of that in other application logs).

mitchellporter commented 8 years ago

I'm curious about this as well. If you look at Loggly's documentation for setting up nodejs it makes no mention of winston-loggly. It only tells you to use Loggly's own fork of this package called winston-loggly-bulk.

My guess is that Loggly is forcing the bulk option for the exact performance reasons you mentioned.

@eschwartz did you make any progress on this?

mitchellporter commented 8 years ago

Here's an update straight from Loggly themselves: https://github.com/winstonjs/winston-loggly/issues/6#issuecomment-229098530

They added bulk support to deal with memory issues and sending so many individual requests. So it looks like this package should not be used, and winston-loggly-bulk should be used instead.