kostya / eye

Process monitoring tool. Inspired from Bluepill and God.
MIT License
1.19k stars 86 forks source link

It's not working when multiple applications #171

Closed emn178 closed 8 years ago

emn178 commented 8 years ago

Hi,

I have two projects, and try to run two sidekiq like this:

rails_env = ENV['APP_ENV'] || 'development'

Eye.config do
  logger "/tmp/eye.log"
end

Eye.application 'app' do
  app_dir = '/Users/emn178/Projects/app'
  working_dir app_dir

  process :sidekiq do
    log_file = "#{app_dir}/log/sidekiq.log"
    pid_file = "#{app_dir}/tmp/pids/sidekiq.pid"
    pid_file 'tmp/pids/sidekiq.pid'
    start_command "bundle exec sidekiq -e #{rails_env} -P #{pid_file} -L #{log_file} -d"
    stop_command "bundle exec sidekiqctl stop #{pid_file}"
  end
end

Eye.application 'app2' do
  app_dir = '/Users/emn178/Projects/app2'
  working_dir app_dir

  process :sidekiq do
    log_file = "#{app_dir}/log/sidekiq.log"
    pid_file = "#{app_dir}/tmp/pids/sidekiq.pid"
    pid_file 'tmp/pids/sidekiq.pid'
    start_command "bundle exec sidekiq -e #{rails_env} -P #{pid_file} -L #{log_file} -d"
    stop_command "bundle exec sidekiqctl stop #{pid_file}"
  end
end

I put eye.rb and run it in app project. app2 sidekiq can not start and always crashed. eye log like this:

08.05.2016 17:58:12 INFO  -- [app:sidekiq] executing: `bundle exec sidekiq -e development -P /Users/emn178/Projects/app/tmp/pids/sidekiq.pid -L /Users/emn178/Projects/app/log/sidekiq.log -d` with start_timeout: 15.0s, start_grace: 2.5s, env: '' (in /Users/emn178/Projects/app)
08.05.2016 17:58:12 INFO  -- [app2:__default__] send_command: monitor
08.05.2016 17:58:12 INFO  -- [app2:__default__] schedule :monitor (reason: monitor by user)
08.05.2016 17:58:12 INFO  -- [app2:__default__] => monitor  (reason: monitor by user)
08.05.2016 17:58:12 INFO  -- [Eye] <= loading: ["/Users/emn178/Projects/app/eye.rb"]
08.05.2016 17:58:12 INFO  -- [app2:__default__] starting async with 0.2s chain monitor []
08.05.2016 17:58:12 INFO  -- [Eye] <= command: load /Users/emn178/Projects/app/eye.rb (0.155443s)
08.05.2016 17:58:12 INFO  -- [app2:sidekiq] schedule :monitor (reason: monitor by user)
08.05.2016 17:58:12 INFO  -- [app2:sidekiq] => monitor  (reason: monitor by user)
08.05.2016 17:58:12 INFO  -- [app2:__default__] <= monitor
08.05.2016 17:58:12 INFO  -- [app2:sidekiq] load_external_pid_file: pid_file not found
08.05.2016 17:58:12 INFO  -- [app2:sidekiq] switch :starting [:unmonitored => :starting] (reason: monitor by user)
08.05.2016 17:58:12 INFO  -- [app2:sidekiq] executing: `bundle exec sidekiq -e development -P /Users/emn178/Projects/app2/tmp/pids/sidekiq.pid -L /Users/emn178/Projects/app2/log/sidekiq.log -d` with start_timeout: 15.0s, start_grace: 2.5s, env: '' (in /Users/emn178/Projects/app2)
08.05.2016 17:58:14 INFO  -- [app:sidekiq] sleeping for :start_grace 2.5
08.05.2016 17:58:14 INFO  -- [app2:sidekiq] sleeping for :start_grace 2.5
08.05.2016 17:58:17 INFO  -- [app:sidekiq] load_external_pid_file: process <46627> from pid_file found and running (identity: ok) (ruby /Users/emn178/.rvm/gems/ruby-2.3.0/bin/sidekiq -e development -P /Users/emn178/Projects/app/tmp/pids/sidekiq.pid -L /Users/emn178/Projects/app/log/sidekiq.log -d)
08.05.2016 17:58:17 INFO  -- [app:sidekiq] switch :started [:starting => :up] (reason: monitor by user)
08.05.2016 17:58:17 INFO  -- [app:sidekiq] <= monitor
08.05.2016 17:58:17 INFO  -- [app2:sidekiq] load_external_pid_file: process <46629> from pid_file found and running (identity: ok) (ruby /Users/emn178/.rvm/gems/ruby-2.3.0/bin/sidekiq -e development -P /Users/emn178/Projects/app2/tmp/pids/sidekiq.pid -L /Users/emn178/Projects/app2/log/sidekiq.log -d)
08.05.2016 17:58:17 INFO  -- [app2:sidekiq] switch :started [:starting => :up] (reason: monitor by user)
08.05.2016 17:58:17 INFO  -- [app2:sidekiq] <= monitor
08.05.2016 17:58:22 WARN  -- [app2:sidekiq] check_alive: process <46629> not found
08.05.2016 17:58:22 INFO  -- [app2:sidekiq] switch :crashed [:up => :down] (reason: crashed)

It can work if I run the the start command manually. in /Users/emn178/Projects/app

bundle exec sidekiq -e development -P /Users/emn178/Projects/app/tmp/pids/sidekiq.pid -L /Users/emn178/Projects/app/log/sidekiq.log -d

/Users/emn178/Projects/app2

bundle exec sidekiq -e development -P /Users/emn178/Projects/app2/tmp/pids/sidekiq.pid -L /Users/emn178/Projects/app2/log/sidekiq.log -d

sidekiq log in local does not make sense, but I saw something strange things in sidekiq log in staging server. It seems app2 used app bundle to run sidekiq and failed.

...
/home/emn178/app2/releases/20160508083837/config/initializers/date_picker.rb:1:in `<top (required)>'
/home/emn178/app/shared/bundle/ruby/2.3.0/gems/railties-4.2.3/lib/rails/engine.rb:652:in `block in load_config_initializer'
...

eye version is 0.8

kostya commented 8 years ago

are you using rvm or rbenv?

emn178 commented 8 years ago

Hi,

I'm using rvm. I solved my problem by moving eye to global instead of run by a project bundle. You can close this issue if this is normal behavior.

Thanks.

kostya commented 8 years ago

rvm is well known - hard to use with daemonizing systems (eye, crontab. ...), and gives many bugs like yours, imo rbenv is better, it easier to configure because you only need to set some env variables.

sarbadaj commented 5 years ago

@kostya I am facing the same problem. I have used eye in two different apps. I am trying to load the eye from them. But I am also getting the same problem as mention as above.