kostya / eye

Process monitoring tool. Inspired from Bluepill and God.
MIT License
1.19k stars 86 forks source link

daemonize true and pid_file #187

Closed grimm26 closed 7 years ago

grimm26 commented 7 years ago

When daemonize is set to true, is the managed process supposed to manage the pid_file or is eye? I have a process that eye is managing that we were originally having it manage its own pid_file and it does not daemonize. So the eye config has daemonize true. We use a restart_command to send a signal to tell the process to gracefully restart. With it managing the pid_file, we hit a problem where eye was not happy that something else touched the pid_file after the restart and would reset it and then look for the old PID, which is now gone.

So I disabled the process managing its own pid_file but eye loses track of the process on restart anyway because it keeps trying to find the process in the pid_file that it originally wrote.

Is a restart_command that does a graceful replacement of the process incompatible with daemonize true?

kostya commented 7 years ago

when daemonize: true, eye fully managed pid_file, and not happy when someone replace it. why process which not daemonized managed it pid_file, what the goal of it?

Looks like unicorn behavour, it also managed it pid_file and change/restart it sometime, but unicorn self-daemonized: https://github.com/kostya/eye/blob/master/examples/unicorn.eye

kostya commented 7 years ago

you can reset check pid file identity by check_identity = false, but this is quite hack, and all should work with correct config.

grimm26 commented 7 years ago

The goal of it managing the pid_file is so that when you send it a USR2 and it replaces itself, the new PID is recorded. I will try check_identity false as well as starting the process with a nohup.

kostya commented 7 years ago

this is how unicorn also works, but it have self-daemonization mode. check maybe your process also can do it.

grimm26 commented 7 years ago

yeah, this is an in-house app in Go. I just talked to the dev and asked him about having it daemonize itself but that will take some coding time.

kostya commented 7 years ago

there is options(with defaults) around this problem:

    check_identity: true,
    check_identity_period: 60.seconds,
    check_identity_grace: 60.seconds,
    auto_update_pidfile_grace: 30.seconds,
    revert_fuckup_pidfile_grace: 120.seconds

check_identity - check that pid_file point on correct process (compare pid_file ctime and process start_time). check_identity_period - interval of check_identity. check_identity_grace - max possible difference between process start_time and pid_file ctime. auto_update_pidfile_grace - when pid_file updated by someone, and process exists, it accept this update after this interval. revert_fuckup_pidfile_grace - when pid_file updated by someone, but process not exists, it revert pid_file after this interval.

you can see in eye log, what happen with this process.