Implement logrotate for elasticsearch logs

[dpb587]

Currently elasticsearch manages the rotation, but they never get trashed.

[sopel]

I consider this to be an operational bug and propose it as a candidate for the next milestone due to the implied availability/resilience risks accordingly.

[dpb587]

Implemented, tested:

rotating pattern: /app/var/log/*.log /app/var/log/elasticsearch/*.log  forced from command line (7 rotations)
empty log files are rotated, old logs are removed
considering log /app/var/log/elasticsearch-1.log
  log needs rotating
considering log /app/var/log/kibana-1.log
  log needs rotating
considering log /app/var/log/logstash_redis-1.log
  log needs rotating
considering log /app/var/log/lumberjack_redis-1.log
  log needs rotating
considering log /app/var/log/nginx-access.log
  log needs rotating
considering log /app/var/log/redis-1.log
  log needs rotating
considering log /app/var/log/syslog_redis-1.log
  log needs rotating
considering log /app/var/log/elasticsearch/process.log
  log needs rotating
considering log /app/var/log/elasticsearch/requests.log
  log needs rotating
rotating log /app/var/log/elasticsearch-1.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/elasticsearch-1.log to /app/var/log/elasticsearch-1.log-20140205
truncating /app/var/log/elasticsearch-1.log
rotating log /app/var/log/kibana-1.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/kibana-1.log to /app/var/log/kibana-1.log-20140205
truncating /app/var/log/kibana-1.log
rotating log /app/var/log/logstash_redis-1.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/logstash_redis-1.log to /app/var/log/logstash_redis-1.log-20140205
truncating /app/var/log/logstash_redis-1.log
rotating log /app/var/log/lumberjack_redis-1.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/lumberjack_redis-1.log to /app/var/log/lumberjack_redis-1.log-20140205
truncating /app/var/log/lumberjack_redis-1.log
rotating log /app/var/log/nginx-access.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/nginx-access.log to /app/var/log/nginx-access.log-20140205
truncating /app/var/log/nginx-access.log
rotating log /app/var/log/redis-1.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/redis-1.log to /app/var/log/redis-1.log-20140205
truncating /app/var/log/redis-1.log
rotating log /app/var/log/syslog_redis-1.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/syslog_redis-1.log to /app/var/log/syslog_redis-1.log-20140205
truncating /app/var/log/syslog_redis-1.log
rotating log /app/var/log/elasticsearch/process.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/elasticsearch/process.log to /app/var/log/elasticsearch/process.log-20140205
truncating /app/var/log/elasticsearch/process.log
rotating log /app/var/log/elasticsearch/requests.log, log->rotateCount is 7
dateext suffix '-20140205'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding logs to compress failed
glob finding old rotated logs failed
copying /app/var/log/elasticsearch/requests.log to /app/var/log/elasticsearch/requests.log-20140205
truncating /app/var/log/elasticsearch/requests.log

[dpb587]

This also renames our /app/var/log/elasticsearch/* log files to be more predictable as process.log and requests.log