datalust / seq-forwarder

Local collection and reliable forwarding of log data to Seq
Apache License 2.0
57 stars 15 forks source link

Is there a seq-forwarder deserializer pacakge #64

Closed bigswede74 closed 1 year ago

bigswede74 commented 1 year ago

Is there a deserializer similar to serilog-formatting-compact-reader for the standard serilog json formatter. In other words consuming/deserialize seq-forwarder messages.

nblumhardt commented 1 year ago

Not that I know of currently. I'm up for trying to brainstorm a solution, though, if you can let me know a bit more about your goals and constraints.

bigswede74 commented 1 year ago

Hopefully to make a long story short, I have devices that are deployed to customer sites that do not have internet connectivity and require monitoring. The problem is collecting the device application logs in this environment. The preferred solution would be to have a centralized seq server however we have an internet connectivity limitation. The first solution was to deploy a seq server per installation, however this proved to be too much overhead and maintenance on a per customer basis. The next solution was to build a small .net core web api that can ingest Serilog.Sinks.Seq messages in the CLEF format and write them to disk using Serilog.Sinks.File. This solution can break the logs down by device and the technician can collect the logs folder when there are issues. This solution works however the problem I am trying to solve is changing the seq serverUri while the application is running and the logger has already been built. This configuration is not reloaded when it is updated in the appsettings.json file so I turned to the seq-forwarder which can be updated at runtime by editing the config file \ProgramData\Seq\Forwarder\SeqForwarder.json. The problem with this solution is that the seq-forwarder does not output the logs in CLEF which the web api consumes. So the root of the issue is that I need to update the Seq serverUri at runtime without restarting the application so the logs are sent to the correct endpoint. I appreciate the help and welcome any suggestions.

nblumhardt commented 1 year ago

Thanks for the clarification! I just hatched a bit of a novel solution, let me know if this does the job :-)

Step 1. Install Serilog.Sinks.Map

dotnet add package Serilog.Sinks.Map

The "map" sink handles opening/closing/flushing other sinks for us; it normally does this to maintain a set of different sinks for different outputs, but we can take advantage of the same behavior to have it manage the transition from one Seq server URL to another.

Step 2. Map from the server URL to Seq sink

Log.Logger = new LoggerConfiguration()
    .WriteTo.Map(
        _ => GetServerUrl(),
        (url, wt) => wt.Seq(url),
        sinkMapCountLimit: 1)
    .CreateLogger();

The GetServerUrl() function needs to be written - it should just return a string representing the current Seq server URL.

And that's it! When the URL changes (however that occurs) the Seq sink will be flushed and reloaded.

Let me know how you go, hope this helps!

Nick

bigswede74 commented 1 year ago

Thanks Nick, that worked like a charm!

Again thanks for taking the time.

Brad

nblumhardt commented 1 year ago

Awesome! You're welcome 👍