Make it possible to set max log size

sematext / logagent-js

Extensible log shipper with input/output plugins, buffering, parsing, data masking, and small memory/CPU footprint

https://sematext.com/logagent

Apache License 2.0

389 stars 79 forks source link

Make it possible to set max log size #20

Closed otisg closed 8 years ago

otisg commented 8 years ago

Maybe we should have --max-log-size

Maybe we should also have an option that lets one control what happens with logs that are bigger than --max-log-size. Such logs could:

just get dropped, or
get trimmed to --max-log-size

I assume we should log such situations, so one can see when they have this problem.

megastef commented 8 years ago

+1 for cli and config entry

Currently this is possible with the features from logsene-js (used in logagent-js) by setting environment variables:

export LOGSENE_MAX_MESSAGE_FIELD_SIZE=245760 # in bytes

The above setting would trims "message" field to 240K. Actually 240K is the new default in logsene-js.

Do we need the option to drop the message? I think its much better to trim it to the right size according to the field size (schema/field limitations) used in backends.

otisg commented 8 years ago

OK. No need to add drop until we see the real need.