File based provider?

[sommmen]

Hiya,

This seems exactly what a was looking for. Only I have a File sink setup instead of sql. Would it be feasible to implement a file-based provider for serilog-ui?

An excerpt of my log file;

[10:20:51 INF] Sulis python based parser: null {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.Windows.ProcessRunner"}
[10:20:51 INF] Process Sulis python based parser exited with exit code 0 at 19-2-2021 10:20:51 {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.Windows.ProcessRunner"}
[10:20:51 INF] Done running {"Name":"Sulis python based parser","WaitForExit":true,"DurationMilliseconds":882.0070000000001,"TimeOutMilliseconds":0,"ExitedWhenWaitingForTimeout":false,"exitLogInfo":{"ExitCode":0,"ExitTime":"2021-02-19T10:20:51.9647150+01:00"}} {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.Windows.ProcessRunner"}
[10:30:31 INF] Skipping download of remote file /StaticFiles/TemboAnalyticsLogs_20201023.zip, local file ...\Tembo Paper - KAMPEN (NL) - Sulis_P02378_Line1\TDCNL_0045~TemboAnalyticsLogs_20201023.zip was up to date, Local file size: 7116022 vs remote file size: 7116022 {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.HangFireJobs.RemoteDataFilePuller"}
[10:30:31 INF] Skipping download of remote file /StaticFiles/TemboAnalyticsLogs_20201026.zip, local file ...\Tembo Paper - KAMPEN (NL) - Sulis_P02378_Line1\TDCNL_0045~TemboAnalyticsLogs_20201026.zip was up to date, Local file size: 7393688 vs remote file size: 7393688 {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.HangFireJobs.RemoteDataFilePuller"}
[10:30:31 INF] Skipping download of remote file /StaticFiles/TemboAnalyticsLogs_20201027.zip, local file ...\Tembo Paper - KAMPEN (NL) - Sulis_P02378_Line1\TDCNL_0045~TemboAnalyticsLogs_20201027.zip was up to date, Local file size: 7628661 vs remote file size: 7628661 {"SourceContext":"Tdc.GenesisMachine.RemoteData.Shared.HangFireJobs.RemoteDataFilePuller"}

The logging config is as follows:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information",
      "System.Net.Http.HttpClient": "Information",
      "Hangfire": "Information"
    }
  },
  "Serilog": {
    "Using": [ "Serilog.Sinks.Console" ],
    "WriteTo": [
      {
        "Name": "Console",
        "outPutTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}"
      },
      {
        "Name": "File",
        "Args": {
          "path": "%CUROUTDIR%\\..\\Logs\\RemoteData.WorkerService.txt",
          "rollingInterval": "Day",
          "rollOnFileSizeLimit": true,
          "retainedFileCountLimit": 31,
          "fileSizeLimitBytes": 5242880,
          "outPutTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}"
        }
      }
    ]
  }

I could of course change the template to something that would be easier to parse. I'd just like your opinion if something like this would be feasible and perhaps I take a crack at it.

Adding another provider for sql server for a fancy log viewer seems a bit too much consider the scope of my project. Although perhaps that's a common solution.

In anyway I want to keep file based logging.

[mo-esmp]

@sommmen you need to implement IDataProvider interface for the file sink. Please check out other providers to see how they are implemented.

There are several issues that need to be considered:

• Paging: how to calculate the total number of logs? (logs are saved in multiple files)
• How to read logs via multiple files (for example, the log file for today contains 5 lines, and the log file last day contains 200 lines and for each page10 records should be displayed. now 5 lines from today's log file and 5 logs from last day's log file should be fetched)
• If there were 15 pages and the user clicked on page 7, which file(s) should be opened to read logs?

[sommmen]

Hiya, Thanks - seems solid advice.

When I have some time I may check around i just cloned the repo. Perhaps this is just not a pragmatic approach - I also realized i have log files for my Server and my seperate workerservice app so i'd even have to have an option to aggregate those.

[mo-esmp]

Please share your thoughts and finding with me so we could implement the file provider together. Looking forward to hearing from you.

[sommmen]

Please share your thoughts and finding with me so we could implement the file provider together. Looking forward to hearing from you.

Hiya - well i'll be working on this in my free time a bit and as part of a project for my company, quite busy atm, so i won't be able to just pour in a good weekend of development i'm afraid. Anyhow i'll fiddle around a little bit now. I just cloned the repo locally and made a FileDataProvider - but i'm unfamiliar with how contributing with github works. Should I fork the repo and make a PR at the end or can i make a branch on the main repo? I'm used to just creating a feature branch and then merging that into master at some point.

[mo-esmp]

• First fork the repo
• Checkout dev branch
• Create a feature branch
• Merge feature branch into dev
• Send a pull request