Log Monitor Feedback and Future Features

[lucillex]

Hi everyone! I'm Lucille, and I'm a PM on the Windows Container team working on Log Monitor.

Our team would love to hear from the community on how to improve Log Monitor. We are currently planning for the next versions of Log Monitor and would like to hear feedback.

Here are some guiding questions we would love to hear your thoughts on:

1. How are you currently using Log Monitor? What problem is it solving for you?
2. What issues are you facing surrounding Log Monitor?
3. What features would you like to see?

[mloskot]

1. How are you currently using Log Monitor? What problem is it solving for you?

In Docker images used to deploy custom web services:

CMD C:\LogMonitor.exe /Config C:\LogMonitorConfig.json C:\ServiceMonitor.exe My.Custom.Service

or just

CMD C:\LogMonitor.exe /Config C:\LogMonitorConfig.json

to monitor *.log files in configured location and stream out the log to container stdout.

2. What issues are you facing surrounding Log Monitor?

None.

3. What features would you like to see?

Setting to add log file name to every line streamed out

[israelvaldez]

How are you currently using Log Monitor?

ENTRYPOINT ["cmd", "/S", "/C", "powershell","-command", "./Scripts/RunIisSetup.ps1", "&&", "C:/LogMonitor/LogMonitor.exe", "ServiceMonitor.exe", "w3svc"]

What problem is it solving for you?

Be able to expose IIS Logs and custom logs from our IIS and Windows Services container images

What issues are you facing surrounding Log Monitor?

Should have more integration with Azure Container Insights, so that we could filter out logs for each type of log that we'd be exposing using a single config file, right now we get all logs in different lines with no details on how to separate them, our log files would have new names over time, for example: C:\logs\log-2022-11-16-001.txt and have a rollover functionality so what was mentioned in the previous post would make it almost possible to create kusto queries to filter out the data but would probably make it a more expensive query than a simple starts with, with the type of log that we're filtering, not the file name of where the log line is coming from

What features would you like to see?

Mentioned in the issues question

[Ketsueki2b]

Same for me: I wouldlike a native way to separate logs in order to categorize them. It could be specific headers in brackets (example: [Provider_Source]), separated objects or even files... I don't know, but something I could separate thanks to PowerShell from the host machine or just to have a better view from Admin Center (which already activates the timestamp parameter by default btw). Thanks!

[Ketsueki2b]

Something similar of what you did in the last update for the file provider/source could be nice for others providers/sources too: [Log File: LogFiles\W3SVC1\u_ex221125_x.log] ; That could help a lot to separate all logs as a quick and MVP solution. Thanks!

[lucillex]

@israelvaldez @Ketsueki2b Thanks for the feedback! I'm interested in if tagging the log source in Azure Monitor would help with filtering and categorizing logs?

We've also started a discussion around with supporting JSON outputs here that we would appreciate feedback on.

[paultjohnson]

FileMonitor v2.1 and 2.0

How are you currently using Log Monitor? What problem is it solving for you? SHELL ["C:\LogMonitor\LogMonitor.exe", "powershell.exe"] ENTRYPOINT Start-Service WMSVC; C:\ServiceMonitor.exe w3svc; I am using LogMonitor to stream EventLogs to stdout in a Windows container for and IIS app.

What issues are you facing surrounding Log Monitor? I am encountering an issue reported here: https://github.com/microsoft/windows-container-tools/issues/40 Which is currently closed and I can see the change applied from that ticket into source code for the exe I'm using. The issue is the event log item contains "%2f" and doesn't get sent to stdout because of this error: WARNING: Failed to render event log event. The event will not be processed. Error: 15029

What features would you like to see? Send event log message containing "&2f" to stdout without manipulating the message contents?

[DmitrySenin]

How are you currently using Log Monitor? What problem is it solving for you?

We went live with Windows Containers in October, 2020 with v1.1 but quickly dropped LogMonitor because it would mess up our logs and by then the project looked to have been abandoned. Now we are trying v2.1.0-rc0 due to the custom formatting feature cause all our logs are json. We are going to monitor three main sources

• EventLog (Application and System)
• ETW (IIS logs)
• File (Application logs)

and format them uniformly as json. Application logs are already json, so we are making sure that Event Logs and ETW are formatted accordingly As for Dockerfile, it looks something like this

ENTRYPOINT ["C:\LogMonitor\LogMonitor.exe", "powershell.exe", "-Command"]
CMD [ "entrypoint.ps1" ]

What issues are you facing surrounding Log Monitor? Originally, logs were mixed up but we didn't log anything. Since we are only adopting v2.1.0, nothing to report yet. But #35 and similar ones are concerning. Hence, we are thinking about writing Application Logs to Event Log but there is a catch (see feature proposals)

What features would you like to see? Since we are seeing a bunch of complains about File Watcher, we are thinking about writing Application Logs to Event Log. But they are already json formatted properly and we don't need any extra formatting whereas other Event logs are not. And there is currently no way to distinguish between "our logs" and something else on Event Log. However, Event Log does have Source, and it would be great to be able to filter by it so that the config would look like this

{
  "LogConfig": {
    "logFormat": "custom",
    "sources": [
      {
        "type": "EventLog",
        "startAtOldestRecord": true,
        "eventFormatMultiLine": false,
        "channels": [
          {
            "name": "system",
            "level": "Information",
            "source": "!=MyApp"
          },
          {
            "name": "application",
            "level": "Error",
            "source": "!=MyApp"
          }
        ],
        "customLogFormat": "{'timestamp':'%TimeStamp%','message':'%Message%'}|json"
      },
      {
        "type": "EventLog",
        "startAtOldestRecord": true,
        "eventFormatMultiLine": false,
        "channels": [
          {
            "name": "system",
            "level": "Information",
            "source": "=MyApp"
          }
        ],
        "customLogFormat": "%Message%"
      }
    ]
  }
}

[dil-dvinnai]

We're currently evaluating v2.1.0-rc0 because it has the custom message format we need, however it seems like the default output format of the main process is changed from json to xml with logFormat: "custom". Are there plans to allow changing the process output? As all our application logs are jsons, and because of this change, right now only the process output is xml.

[balazspalko]

Firstly, thanks for LogMonitor! It proves to be really helpful for our Team and we still hope we can stick with this for shipping our logs in legacy applications.

How are you currently using Log Monitor? What problem is it solving for you?

We have a containerized web application running in IIS which is hosted in AWS ECS and we would like to stream our logs to cloudwatch through the standard output. Our application logs to files in JSON format. An example of a log entry in our logfile is:

{"Timestamp":"2023-11-30T11:50:52.8809943+00:00","Level":"Debug","MessageTemplate":"Attempting to load colours xml","Properties":{"SourceContext":"Helpers.CustomLogger","Scope":[{"SessionId":"51706ddd710e4d1182d26f53fe1c7458", "PageRequest":"/Default.asp"}],"Application":"WebAPI"}}

As you can see, we our logging structured logs into our files. This is something we really want so we can leverage what Cloudwatch can provide

What issues are you facing surrounding Log Monitor?

The issue that we faced with LogMonitor is that even though our files have structured JSONs in every line, LogMonitor will wrap our log entries into a stringified field. Our entries in cloudwatch will end up like this:

{
    "Source": "File",
    "LogEntry": {
        "Logline": "{\"Timestamp\":\"2023-11-30T11:50:52.8809943+00:00\",\"Level\":\"Debug\",\"MessageTemplate\":\"Attempting to load colours xml\",\"Properties\":{\"SourceContext\":\"Helpers.CustomLogger\",\"Scope\":[{\"SessionId\":\"51706ddd710e4d1182d26f53fe1c7458\",\"PageRequest\":\"/Default.asp\"}],\"Application\":\"WebAPI\"}}",
        "FileName": "log20231130.txt"
    },
    "SchemaVersion": "1.0.0"
}

As you can see, everything is pushed into LogLine as string and with that we lose information on our structured logs. From this point on it's why more difficult to write cloudwatch queries for listing all the logs that belong to the PageRequest Default.asp.

We have actually found a workaround this in 2.1.0 rc1. We are using a custom log format and use the following logmonitor config:

{
    "LogConfig": {
        "logFormat": "custom",
        "sources": [
            {
                "type": "File",
                "directory": "D:\\Logs\\",
                "filter": "log*.txt",
                "includeSubdirectories": true,
                "customLogFormat": "%Message%"
            }
        ]
 }

This will actually solve our problem on the JSON file logs. However as @dil-dvinnai mentioned in his comments, for some reason, the logs that are actually written to process are serialized as XML. And there's no way to modify that from configuration:

<Log><Source>Process</Source><LogEntry><Logline>Initializing service...</Logline></LogEntry></Log>

Our relevant logs are mostly in files, but it is odd now that part of our logs are sent as json, while logs that are actually written by our entrypoint process to stdout are sent as XML.

What features would you like to see?

Would be nice if custom logging would be part of an official release not just a RC. We're also not sure if this is really the right way to approach our problem. Feels a bit hacky. At the end of the day we just want that our structured JSON log entries in our files are not actually wrapped into a stringified json by logmonitor.

[david-garcia-garcia]

How are you currently using Log Monitor? What problem is it solving for you?

Log Monitor is critical when dockerizing Windows Apps, specially considering that many of them rely on a set of moving pieces (IIS, Scheduled Tasks, Event Viewer, Other subservices) and are not mono purpose or not built with microservices architecture. The EntryPoint is just not sufficient enough to reflect what is going on inside the container. Not just legacy apps, it sometimes makes sense to build monolithic stuff to keep costs down for some type of projects. Even to keep track of the OS health, seeing event viewer output through the container stdout is a must.

What issues are you facing surrounding Log Monitor?

As mentioned by other posters, file based monitoring features are lacking some features.

What features would you like to see?

https://github.com/microsoft/windows-container-tools/issues/168

[KiranMulawad]

Same for me: I wouldlike a native way to separate logs in order to categorize them. It could be specific headers in brackets (example: [Provider_Source]), separated objects or even files... I don't know, but something I could separate thanks to PowerShell from the host machine or just to have a better view from Admin Center (which already activates the timestamp parameter by default btw). Thanks!

By any chance, is this above feature implemented in any of the new versions of LogMonitor. Our team is also looking out for the same, i.e. appending the log source name to the logs relayed back by LogMonitor tool, so that it would be easy for us to categorize the logs based on the their source.