search

serilog-contrib / serilog-sinks-elasticsearch

A Serilog sink that writes events to Elasticsearch
Apache License 2.0
433 stars 196 forks source link

ElasticSearchSink durability - rolling file problem #27

Closed nblumhardt closed 4 years ago

nblumhardt commented 8 years ago

Moved from: https://github.com/serilog/serilog/issues/531

We have set up the ELS sink with durability in our ASP.NET application (running on IIS / Win2012 R2). We have also added a max size on the file, so that it will roll the files if they become too big. Our configuration is as follows:

jarlelin commented 8 years ago

This does not seem to be supported in the code for the rolling file sink, but I would like to be able to set the maximum file size per file, and toggle the sink to roll to a new file upon reaching the max. I would also like to be able to configure the maximum amount of days that the elastic search log shipper retains files for before deleting them.

This would allow me to keep 10 days of logs, each day spread over the amount of files needed based on file size maximum. Is there any support for such a feature on this package? Would you accept a pull request that enables it?

mivano commented 8 years ago

I do not think it is supported at the moment. Open for PR of course.

mhermann commented 7 years ago

We have a very similar use case.

I can see that the alternate rolling file sink actually supports rolling files on date and size. From looking at the code of the log shipper I believe that support for multiple files per day is build into this and if we could have the sink support this as well we would have solved the problem.

I'm thinking of giving this a go and have thought about a couple of possible solutions:

  1. Allow injection in of a custom sink either in the ElasticSearchOptions or as a parameter in the LoggerConfiguration extension. This would then be used in DurableElasticSearchSink which would mean that the DurableElasticSearchSink should support a generic sink as well as the default constructor using the RollingFileSink

  2. Make a new Nuget package forking the existing Serilog.Sinks.Elasticsearch called Serilog.Sinks.Elasticsearch.RollingFileAlternate. This would require maintaining a separate repo but would not require any changes to the existing sink.

Any other ideas or input on how this can be achieved?

mivano commented 4 years ago

Old issue, cleaning up. Still facing this issue, feel free to reopen.

mivano commented 4 years ago

Old issue, cleaning up. Still facing this issue, feel free to reopen.