Is there a possibility to change to OUTPUT file to a custom /var/log destination

[avwsolutions]

Is there a possibility to change to OUTPUT file to a custom /var/log destination ?

In some cases you want to use for example /var/log/sysmon instead of the standard syslog, since it will overload the syslog for Linux admins. Additionally we can ensure that logging tools easily can harvest different formats without running to much reg matches.

[juju4]

You can do with rsyslog. for example with https://github.com/juju4/ansible-sysmon/blob/master/templates/rsyslog-sysmon.conf.j2 at least until a native option to directly log to file.

[hillu]

Adding that plain file backend is a good idea because having systemd-journald(8) handle possibly vast numbers of messages does incur quite some cost in CPU overhead.

[kesheldr]

Possibly; I will add it to the list of suggestions to debate. Would it be better to build log rotation into Sysmon or provide a method that logrotate could use to rotate for it?

[juju4]

IMHO better to use external logrotate for rotation with just ensuring a HUP signal forces reopening log file.

But that does not prevent to have option to log a file directly without passing by rsyslog. most SIEM connector/forwarder handles that without issue. avoid rsyslog/journald overhead. Amusingly enough, not the Microsoft Monitoring Agent or Azure Monitoring Agent for now, I believe.

[hillu]

A problem with relying on logrotate (the implementation, not the concept) is that it is usually not run very often. (The presence of a /etc/cron.daily/logrotate script hints at once per day). For high-volume telemetry that is going to be shipped off to another system to be processed there, this is not good enough.

[juju4]

agreed on rotation frequency. you can likely change it from cron.daily to cron.hourly but may have side impacts. or just add extra call like https://jira.atlassian.com/browse/HCPUB-546 (care if dateext is enabled as filename conflict)

[ssi0202]

@juju4 will the solution to mention result in the file being a pure xml formattet log data? the reason i ask is that im looking at how one can ship sysmon log data off the endpoint having a xml in json within syslog file presents a bit of a problem when the data then have to be ingested in xyz log managment platform. the data is there but getting it in a good workable datashema takes extra work (its doable no doubt) the datachema and thus fieldnames etc is a pre requsite to easily run sigma rules against the sysmon data

[juju4]

at this point, there is no solution implementation so it is still opened but unlikely, logs are written per event, most often line by line which means log file can't be a full valid json or xml which both have start/end marker.

Most SIEM/logging platforms have parse xml operators (same than json) https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/parse-xmlfunction https://docs.splunk.com/Documentation/SplunkCloud/latest/Search/Extractfieldswithsearchcommands (spath, xmlkv...) https://help.sumologic.com/05Search/Search-Query-Language/01-Parse-Operators/07-Parse-XML-Formatted-Logs https://www.elastic.co/guide/en/logstash/current/plugins-filters-xml.html

in cli, xmllint or sysmonLogView are the main option but sysmonLogView is not very customizable IMHO and xmllint requires a fully valid xml input so needs some commandline-fu for processing. example

head -1 /var/log/sysmon/sysmon.log | sed 's/^.* sysmon: //' | xmllint --xpath "concat(/Event/EventData/Data[@Name='CommandLine']/text(), '|', /Event/EventData/Data[@Name='ParentCommandLine'
]/text())" -

less straightforward than jq and json.