Integrate a mechanism for receiving notifications when an event log is discarded.

[AmaProject]

Hello,

I'm using Serilog to send syslog messages over UDP or TCP, and I would like to set up a mechanism to be notified whenever an event log is discarded. This could happen due to various reasons such as network issues, queue's errors, or other unforeseen circumstances.

Is there a way to implement a callback or notification mechanism that would be triggered when an event log is dropped or discarded by the logging system? This would help me ensure that no important log entries are missed and allow me to take appropriate action when needed.

For exemple a callback that would be invoked when EmitBatchAsync of SyslogTcpSink/SyslogUdpSink encounters a SocketException, or when PeriodicBatchingSink discards a logEvent during Emit would be great.

I can provide you with an initial implementation that we can refine if you're interested in pursuing this request.

Thank you.

[nblumhardt]

Hello! Thanks for the suggestion and the offer.

As a first step, you should investigate implementing auditing for the sink in question. Auditing uses the same ILogEventSink mechanism as usual, but works in a mode that propagates exceptions.

Step two, once your sink implements auditing, is to implement a sink wrapper using LoggerSinkConfiguration.Wrap(); the configuration of the wrapper sink should include a) the audit-enabled sink to write to, and b) the fallback sink it can forward events to when the auditing sink throws exceptions.

Auditing doesn't currently work directly with batching, but there are a lot more issues (retry policies and so on) that will make this complicated to design atop a batched sink. The syslog protocols are not batched protocols, so this shouldn't be too much of a limitation.

If your wrapper needs to work asynchronously, you can further add WriteTo.Async around it to keep the work on a background thread, so the final configuration might look like:

Log.Logger = new LoggerConfiguration()
    .WriteTo.Async(asyn =>
        asyn.FallbackPair(
            audited => audited.Syslog(...),
            fallback => fallback.File(...)))
    .CreateLogger();

FallbackPair is the bit you'd implement using LoggerSinkConfiguration.Wrap().

Would love to hear how you go!

[AmaProject]

Thank you for your advice; unfortunately, I'm encountering some issues while trying to implement it.

As a first step, you recommend implementing auditing. I've looked into ensuring compatibility between the TcpSyslog sink and AuditTo. Based on my understanding, I've pinpointed the following aspects :

• Create an extension TcpSyslog for LoggerAuditSinkConfiguration.

• SyslogTcpSink already propagates exceptions through its EmitBatchAsync interface, which seems perfect.

• Develop an implementation similar to PeriodicBatchingSink that would propagate exceptions. However, due to the asynchronous writing, the implementation is not straightforward.

Can you confirm that I have understood your approach ?

This appears for me to be quite complex. Wouldn't it be risky to re-implement PeriodicBatchingSink and miss out on the library's bug fixes and improvements? Unless you plan to subsequently introduce an audit-compatible version into Serilog ? If this is for a personal implementation, wouldn't it be easier to add a simple callback to a derived class of PeriodicBatchingSink ?

The second step seems much simpler, but unfortunately, I can't use it without successfully completing Step 1.

[nblumhardt]

Hi! I think the subtleties here are around the batching aspect.

Syslog isn't a batched protocol; the current syslog sinks I know of are using batching as a mechanism to enable retry, which isn't really what you want in your scenario, because if the attempt to emit a syslog message fails, you want to divert it to the secondary sink.

The approach I'm suggesting looks more like:

• Create an extension TcpSyslog for LoggerAuditSinkConfiguration. Ideally, the sink returned from this would be the instance of SyslogTcpSink, but it doesn't currently implement ILogEventSink directly, so you'll need to:
  • return a BatchedSinkAdaptor (your class) which implements ILogEventSink and forwards Emit as single-event batches to SyslogTcpSink, or
  • PR a change to the Syslog sink that adds auditing and support for the non-batched interface

From there, all of the work is in your "FallbackPair" wrapper sink, completely separate from the syslog sink implementation.

Let me know if this clarifies things a bit :) HTH!