mesosphere / chaos

A lightweight framework for writing REST services in Scala.
Apache License 2.0
250 stars 36 forks source link

Allow logging to a file via a cli flag #53

Open SEJeff opened 8 years ago

SEJeff commented 8 years ago

The mesosphere marathon package for redhat logs via logger, which gets hoovered up into the systemd journal on RHEL/CentOS 7 or Fedora. It would be really nice to instead send this to a specific file in addition to the systemd journal.

Since marathon uses this library to do logging, the issue seems to be here and not in marathon directly. I've added this systemd service as a stopgap so we can use splunk to read marathon logs:

[Unit]
Description=Hack to log marathon logs to a file
After=network-online.target
Wants=network.target

[Service]
ExecStart=/bin/bash -c "/usr/bin/journalctl --quiet -lfu marathon >> /var/log/marathon/marathon.log"
Restart=always
RestartSec=1

[Install]
WantedBy=multi-user.target

And I've added Requires=marathon-logger.service to the marathon systemd unit to start marathon-logger on demand. Would it be possible to have chaos, and as a nice side effect, marathon support something like:

--log_level=info --log_file=/var/log/marathon/marathon.log

A simple logrotate config like this could be used so that chaos wouldn't natively need to rotate any logs:

/var/log/marathon/*.log {
    daily
    compress
    missingok
    rotate 30
    notifempty
    copytruncate
    delaycompress
}
jimonreal commented 8 years ago

I have made a few changes to be able to choose the facility using an env var, and if not set use the default "user". Later I want to change that for a more parameter solution, using what you propose above, and been able to choose between syslog, stdout and any other logging driver.

Here is what a would apply first:

diff --git a/bin/marathon-framework b/bin/marathon-framework
index 85858a7..7a6d333 100755
--- a/bin/marathon-framework
+++ b/bin/marathon-framework
@@ -20,6 +20,7 @@ self="$(cd "$(dirname "$0")" && pwd -P)"/"$(basename "$0")"
 marathon_jar="$self"
 conf_dir=/etc/marathon/conf
 conf_file=/etc/default/marathon
+syslog_facility=${MARATHON_SYSLOG_FACILITY:-user}

 function main {
   if [[ ${1:-} = --jar ]]
@@ -60,7 +61,7 @@ function load_options_and_log {
   set -o allexport
   [[ ! -f "$conf_file" ]] || . "$conf_file"
   set +o allexport
-  for env_op in `env | grep ^MARATHON_ | sed -e '/^MARATHON_APP/d' -e 's/MARATHON_//' -e 's/=/ /'| awk '{printf("%s%s ", "--", tolower($1)); for(i=2;i<=NF;i++){printf("%s ", $i)}}'| sed -e 's/ $//'`; do
+  for env_op in `env | grep ^MARATHON_ | grep -v MARATHON_SYSLOG_FACILITY | sed -e '/^MARATHON_APP/d' -e 's/MARATHON_//' -e 's/=/ /'| awk '{printf("%s%s ", "--", tolower($1)); for(i=2;i<=NF;i++){printf("%s ", $i)}}'| sed -e 's/ $//'`; do
     cmd+=( "$env_op" )
   done
   # Default zk and master option
@@ -115,8 +116,8 @@ function run_jar {

 function logged {
   local token="$1[$$]" ; shift
-  exec 1> >(exec logger -p user.info   -t "$token")
-  exec 2> >(exec logger -p user.notice -t "$token")
+  exec 1> >(exec logger -p "${syslog_facility}".info   -t "$token")
+  exec 2> >(exec logger -p "${syslog_facility}".notice -t "$token")
   "$@"
 }
SEJeff commented 8 years ago

@jimonreal That is a good start, but ideally I could still just log to a flat file and have splunk ingest that.