search

logstash-plugins / logstash-output-file

Apache License 2.0
23 stars 53 forks source link

Added Feature: File size base log rotation #85

Open ChrisHubinger opened 4 years ago

ChrisHubinger commented 4 years ago

File size based log rotation

Motivation

In one of our Projects we need to implement log rotation to make sure the disk does not fill up. As it seems not easily possible to simply use logrotate for that purpouse without the risk loosing events I've implemented a file size based rotation directly in this plugin.

Feature already in discussion here: #57

Configuration

Rotation is enabled by setting the parameter file_rotation_size to a value > 0 (Default: 0) and otional set the parameter max_file_rotation to a non negative value to limit the number of created log files. keep_file_extension => true, will prepend the rotation index before the file extension of the output path. Creating files like /path/to/logs/app.0.log /path/to/logs/app.1.log /path/to/logs/app.2.log ...

Implementation notes

The current file size is only checked when rotation is enabled. The cleanup/housekeeping deletes only the last created file when limit has been reached e.g does not scvann folder for further files that may be deleted.

Tests

I added tests to validate the implementation, in a very similar way other features are tested

I'd be very happy to see this feature merged into upstream so that with future releases we do not need to ship our own version of the plugin anymore. Please give me feedback if there is anything further i can do to get this PR accepted.

Thanks to you all for all the work on this great product.

Kind Regards, Christian

ChrisHubinger commented 4 years ago

As I see from the Travic CI errors something in the test setup seeems to have changed from version 5 => 6,7.. as it complains about missing insist method.

I failed with wrapping agent do ... end inside it 'test case' do .. end block so that i can use expect instead.

Any hint for me how i can fix that?

ChrisHubinger commented 4 years ago

@ph @jsvd Is there anything I can do to get feedback to this PR?

oxyrox commented 3 years ago

Are there any plans to get this reviewed?

@ChrisHubinger meanwhile the version has changed to 4.3.0 so you could already rebase and update the version. Hopefully there will be some momentum on this.

We could really use this feature, nice work btw!

taylorjdlee commented 1 year ago

This would be fantastic to add like honestly this feature has been talked about for 7 years but never merged..... a lot of us would like it!

taylorjdlee commented 1 year ago

@jsvd @ph @robbavey @colinsurprenant