ENABLE_LOGROTATION = False with multiple carbon daemons

[jonsulman]

I think the code that supports "logrotate" log rotation is not working under multiple carbon daemons.

log.py:

  def write(self, data):
    if not self.enableRotation:
      if not os.path.exists(self.path):
        self.reopen()
    DailyLogFile.write(self, data)

If you are running multiple daemons, as soon as one daemon notices the log file has rotated it will create a new one. Then for the other daemons os.path.exists(self.path) will be true (seeing the other daemon's newly created file), so they will not reopen their log file and will continue writing to the old file.

As a result you end up with a situation like this:

$ sudo lsof console.log*
COMMAND     PID   USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
carbon-ca 40009 carbon    6u   REG  253,0   585103 4372133 console.log.2016_4_26
carbon-ca 50485 carbon    6u   REG  253,0   798915  574424 console.log.2016_4_23
carbon-re 50563   root    6u   REG  253,0    10090 2728960 console.log

[jonsulman]

550

[jonsulman]

The default settings from carbon.conf.example show how to set ENABLE_LOGROTATE, but don't clearly indicate that since the example is in the [cache] section, it doesn't apply to relay processes unless added to the [relay] section.

[deniszh]

PR is merged, closing