Supervisor / supervisor

Supervisor process control system for Unix (supervisord)
http://supervisord.org
Other
8.46k stars 1.24k forks source link

Rotating supervisord logs #1551

Closed cidrbl0ck closed 1 year ago

cidrbl0ck commented 1 year ago

So Im troubleshooting a problem with log rotation on a server.. what's happening is that when logrotate is called a midnight the supervisord.log is not rotating correctly. I end up with a .log.1, .log.2 and so on. But the supervisord process is not moving to the new log file. If I restart the daemon or run supervisorctl update or restart then it does. I believe the problem lies with logrotate however in researching logging and rotations with supervisord I'm left with more questions than answers. The log files in question: /var/log/cloudbolt/supervisord.log are owned by apache:apache and lsof shows supervisord as the only pid writing to them. Does Supervisord rely upon logrotate or does it perform it's own rotations?

Environment Details: CentOS 7 Supervisord 4.2.2 Logrotate 3.8.6

supervisord.conf:

[unix_http_server]
file=/var/run/cloudbolt/supervisor.sock   ; the path to the socket file

[supervisord]
logfile=/var/log/cloudbolt/supervisord.log ; main log file; default $CWD/supervisord.log
logfile_maxbytes=50MB        ; max main logfile bytes b4 rotation; default 50MB
logfile_backups=10           ; # of main logfile backups; 0 means none, default 10
loglevel=trace                ; log level; default info; others: debug,warn,trace
pidfile=/var/run/cloudbolt/supervisord.pid ; supervisord pidfile; default supervisord.pid
nodaemon=false               ; start in foreground if true; default false
minfds=786068
minprocs=200                 ; min. avail process descriptors;default 200

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/cloudbolt/supervisor.sock ; use a unix:// URL  for a unix socket

[include]
files = /etc/supervisord.d/*.conf

The only file under /etc/supervisord.d/ - jobengine.conf

[program:jobengine]
; Set full path to Job Engine program if using virtualenv
command=python /opt/cloudbolt/jobengine/jobengine.pyc --namespace=worker%(process_num)02d
environment=PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:

directory=/opt/cloudbolt
user=root
numprocs=4
process_name=%(process_num)01d
stdout_logfile=/var/log/cloudbolt/jobengine.log
stderr_logfile=/var/log/cloudbolt/jobengine.log
autostart=true
autorestart=true
startsecs=10
startretries=999

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 590

; Causes supervisor to send the termination signal (SIGTERM) to the whole process group.
stopasgroup=true

; default (999)
priority=999

Thanks!!

mnaberez commented 1 year ago

Does Supervisord rely upon logrotate or does it perform it's own rotations?

supervisord performs its own rotations. It does not use logrotate to do this. Please see the relevant sections in the manual:

http://supervisord.org/logging.html#activity-log-rotation http://supervisord.org/configuration.html#supervisord-section-values