search

enowars / EnoEngine

MIT License
12 stars 2 forks source link

Constrain log length (otherwise ELK dies) #166

Open Trolldemorted opened 3 years ago

Trolldemorted commented 3 years ago

if the disk goes full entire transactions are logged, and then elk goes belly-up

DanielHabenicht commented 3 years ago

see #162 ?

Savallator commented 3 years ago

And what do you want to do when the Disk runs full? It is not like you could recover from that... And regarding the other Issue, if you really want, you could at least use Serilogs to directly log to the logstash. Rolling logfiles are a bad idea and don't help anyone.

Trolldemorted commented 3 years ago

This has nothing todo with #162 or full disks, this is about the length of a single logmessage

Savallator commented 3 years ago

You mentioned the disk running full in your first post?

Savallator commented 3 years ago

I see why it would be nice to truncate overly long log messages, but that won't help when the disk runs full? But anyways, can we not enforce length limits on later stages as well? like in the logstash e.g.?

Trolldemorted commented 3 years ago

You mentioned the disk running full in your first post?

this happens when the disk of the engine runs full, and I don't want full engine disks to kill my Elk (which runs on a different machine) :sad_elk:

Savallator commented 3 years ago

Well, but that could indeed be fixed by cutting out filebeat and logging directly to the logstash. That way logs won't accumulate on the disc.

Trolldemorted commented 3 years ago

sure, but that won't help against other (future) logmessages that will kill elk