search

DarthSim / overmind

Process manager for Procfile-based applications and tmux
MIT License
2.77k stars 79 forks source link

`overmind start` fails if there is more than one process type in the `Procfile` #184

Open jagthedrummer opened 2 weeks ago

jagthedrummer commented 2 weeks ago

I have a simple Procfile that looks like this:

web: bin/rails server -p 3000
worker: bundle exec sidekiq -t 25

When I do overmind start -f Procfile it throws this error:

$ overmind start -f Procfile
system | Tmux socket name: overmind-bullet-train-0T2BtiVZBFDN8R8g7nevu
system | Tmux session ID: bullet-train
system | Listening at ./.overmind.sock
worker | Started with pid 8278...
web    | Started with pid 8277...
worker | /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:283:in `find_spec_for_exe': Could not find 'bundler' (2.4.10) required by your /Users/jgreen/projects/bullet-train-co/bullet_train/Gemfile.lock. (Gem::GemNotFoundException)
worker | To update to the latest version installed on your system, run `bundle update --bundler`.
worker | To install the missing version, run `gem install bundler:2.4.10`
worker |        from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:302:in `activate_bin_path'
worker |        from /usr/bin/bundle:23:in `<main>'
worker | Exited with code 1
web    | Interrupting...
web    | <internal:dir>:223:in `[]': Interrupt
web    |        from /Users/jgreen/projects/bullet-train-co/bullet_train-core/bullet_train-themes-light/bullet_train-themes-light.gemspec:22:in `block (2 levels) in <main>'
web    |        from /Users/jgreen/projects/bullet-train-co/bullet_train-core/bullet_train-themes-light/bullet_train-themes-light.gemspec:21:in `chdir'
web    |        from /Users/jgreen/projects/bullet-train-co/bullet_train-core/bullet_train-themes-light/bullet_train-themes-light.gemspec:21:in `block in <main>'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/specification.rb:2074:in `initialize'
web    |        from /Users/jgreen/projects/bullet-train-co/bullet_train-core/bullet_train-themes-light/bullet_train-themes-light.gemspec:3:in `new'
web    |        from /Users/jgreen/projects/bullet-train-co/bullet_train-core/bullet_train-themes-light/bullet_train-themes-light.gemspec:3:in `<main>'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler.rb:632:in `eval'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler.rb:632:in `eval_gemspec'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler.rb:547:in `block in load_gemspec_uncached'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/shared_helpers.rb:52:in `chdir'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/shared_helpers.rb:52:in `block in chdir'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/shared_helpers.rb:51:in `synchronize'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/shared_helpers.rb:51:in `chdir'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler.rb:546:in `load_gemspec_uncached'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler.rb:532:in `load_gemspec'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/source/path.rb:162:in `load_gemspec'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/source/path.rb:177:in `block in load_spec_files'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/source/path.rb:176:in `each'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/source/path.rb:176:in `load_spec_files'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/source/path.rb:107:in `local_specs'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/source/path.rb:115:in `specs'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/definition.rb:641:in `specs_for_source_changed?'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/definition.rb:626:in `specs_changed?'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/definition.rb:678:in `block in converge_paths'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/definition.rb:677:in `any?'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/definition.rb:677:in `converge_paths'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/definition.rb:136:in `initialize'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/dsl.rb:218:in `new'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/dsl.rb:218:in `to_definition'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/dsl.rb:13:in `evaluate'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/definition.rb:37:in `build'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler.rb:217:in `definition'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler.rb:165:in `setup'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/setup.rb:10:in `block in <top (required)>'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/ui/shell.rb:159:in `with_level'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/ui/shell.rb:111:in `silence'
web    |        from /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/setup.rb:10:in `<top (required)>'
web    |        from <internal:/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
web    |        from <internal:/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
web    |        from /Users/jgreen/projects/bullet-train-co/bullet_train/config/boot.rb:3:in `<top (required)>'
web    |        from bin/rails:3:in `require_relative'
web    |        from bin/rails:3:in `<main>'
web    | Exited with code 0

It seems that the worker process thinks that the bundle command is not available. I originally thought that I was running into this issue: https://github.com/DarthSim/overmind/issues/74 But I don't think that's the case.

I tried to simplify the Procfile to contain only the worker process, and then that works.

Procfile:

# web: bin/rails server -p 3000
worker: bundle exec sidekiq -t 25
$ overmind start -f Procfile
system | Tmux socket name: overmind-bullet-train-b84YAxs2q22IKoNLyl_ZV
system | Tmux session ID: bullet-train
system | Listening at ./.overmind.sock
worker | Started with pid 9519...
worker |
worker |
worker |                m,
worker |                `$b
worker |           .ss,  $$:         .,d$
worker |           `$$P,d$P'    .,md$P"'
worker |            ,$$$$$b/md$$$P^'
worker |          .d$$$$$$/$$$P'
worker |          $$^' `"/$$$'       ____  _     _      _    _
worker |          $:    ',$$:       / ___|(_) __| | ___| | _(_) __ _
worker |          `b     :$$        \___ \| |/ _` |/ _ \ |/ / |/ _` |
worker |                 $$:         ___) | | (_| |  __/   <| | (_| |
worker |                 $$         |____/|_|\__,_|\___|_|\_\_|\__, |
worker |               .d$$                                       |_|
worker |
worker |
worker | 2024-06-19T20:48:00.671Z pid=9595 tid=byj INFO: Booted Rails 7.1.3.4 application in development environment
worker | 2024-06-19T20:48:00.671Z pid=9595 tid=byj INFO: Running in ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin23]
worker | 2024-06-19T20:48:00.671Z pid=9595 tid=byj INFO: See LICENSE and the LGPL-3.0 for licensing details.
worker | 2024-06-19T20:48:00.671Z pid=9595 tid=byj INFO: Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org
worker | 2024-06-19T20:48:00.671Z pid=9595 tid=byj INFO: Sidekiq 7.2.4 connecting to Redis with options {:size=>10, :pool_name=>"internal", :url=>nil}
worker | 2024-06-19T20:48:00.675Z pid=9595 tid=byj INFO: Sidekiq 7.2.4 connecting to Redis with options {:size=>5, :pool_name=>"default", :url=>nil}
worker | 2024-06-19T20:48:00.770Z pid=9595 tid=byj INFO: Starting processing, hit Ctrl-C to stop

Likewise, if I leave only the web process enabled, it also works.

web: bin/rails server -p 3000
# worker: bundle exec sidekiq -t 25
$ overmind start -f Procfile.dev
system | Tmux socket name: overmind-bullet-train-BVFVaQIQ-qUq1p_rVSr8y
system | Tmux session ID: bullet-train
system | Listening at ./.overmind.sock
web    | Started with pid 10404...
web    | => Booting Puma
web    | => Rails 7.1.3.4 application starting in development
web    | => Run `bin/rails server --help` for more startup options
web    | [10470] Puma starting in cluster mode...
web    | [10470] * Puma version: 6.4.2 (ruby 3.3.2-p78) ("The Eagle of Durango")
web    | [10470] *  Min threads: 5
web    | [10470] *  Max threads: 5
web    | [10470] *  Environment: development
web    | [10470] *   Master PID: 10470
web    | [10470] *      Workers: 4
web    | [10470] *     Restarts: (✔) hot (✖) phased
web    | [10470] * Preloading application
web    | [10470] * Listening on http://127.0.0.1:3000
web    | [10470] * Listening on http://[::1]:3000
web    | [10470] Use Ctrl-C to stop
web    | [10470] - Worker 0 (PID: 10998) booted in 0.01s, phase: 0
web    | [10470] - Worker 1 (PID: 10999) booted in 0.0s, phase: 0
web    | [10470] - Worker 2 (PID: 11000) booted in 0.0s, phase: 0
web    | [10470] - Worker 3 (PID: 11001) booted in 0.0s, phase: 0

If I try to run two web processes on different ports it fails in a different way:

web: bin/rails server -p 3000
another: bin/rails server -p 3001
$ overmind start -f Procfile
system  | Tmux socket name: overmind-bullet-train-ay1tK3xF69A-olvz7IAcZ
system  | Tmux session ID: bullet-train
system  | Listening at ./.overmind.sock
web     | Started with pid 13889...
another | Started with pid 13890...
another | You must use Bundler 2 or greater with this lockfile.
another | Exited with code 20
web     | Interrupting...
web     | <internal:/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `set_encoding': Interrupt
web     |       from <internal:/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
web     |       from <internal:/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
web     |       from /Users/jgreen/projects/bullet-train-co/bullet_train/config/boot.rb:4:in `<top (required)>'
web     |       from bin/rails:3:in `require_relative'
web     |       from bin/rails:3:in `<main>'
web     | Exited with code 0

And if I try to run two worker instances:

worker: bundle exec sidekiq -t 25
another: bundle exec sidekiq -t 25
$ overmind start -f Procfile.dev
system  | Tmux socket name: overmind-bullet-train-otygUIkLncCD2_324MX4m
system  | Tmux session ID: bullet-train
system  | Listening at ./.overmind.sock
worker  | Started with pid 14680...
another | Started with pid 14681...
another | /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:283:in `find_spec_for_exe': Could not find 'bundler' (2.4.10) required by your /Users/jgreen/projects/bullet-train-co/bullet_train/Gemfile.lock. (Gem::GemNotFoundException)
another | To update to the latest version installed on your system, run `bundle update --bundler`.
another | To install the missing version, run `gem install bundler:2.4.10`
another |       from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:302:in `activate_bin_path'
another |       from /usr/bin/bundle:23:in `<main>'
another | Exited with code 1
worker  | Interrupting...
worker  | Exited with code 1

If I put the worker process before the web process:

worker: bundle exec sidekiq -t 25
web: bin/rails server -p 3000
$ overmind start -f Procfile
system | Tmux socket name: overmind-bullet-train-KtpVKlgz9c_HLD_dy230l
system | Tmux session ID: bullet-train
system | Listening at ./.overmind.sock
web    | Started with pid 17074...
worker | Started with pid 17073...
web    | You must use Bundler 2 or greater with this lockfile.
web    | Exited with code 20
worker | Interrupting...
worker | Exited with code 1

It seems like maybe the first process gets the same "environment" (not sure this is the right word for this) as the main shell that I'm in, but that subsequent processes somehow get something different.

With the original web/worker Procfile I can run it just fine with foreman:

web: bin/rails server -p 3000
worker: bundle exec sidekiq -t 25
$ foreman start -f Procfile
15:55:35 web.1    | started with pid 16202
15:55:35 worker.1 | started with pid 16203
15:55:36 web.1    | => Booting Puma
15:55:36 web.1    | => Rails 7.1.3.4 application starting in development
15:55:36 web.1    | => Run `bin/rails server --help` for more startup options
15:55:37 web.1    | [16202] Puma starting in cluster mode...
15:55:37 web.1    | [16202] * Puma version: 6.4.2 (ruby 3.3.2-p78) ("The Eagle of Durango")
15:55:37 web.1    | [16202] *  Min threads: 5
15:55:37 web.1    | [16202] *  Max threads: 5
15:55:37 web.1    | [16202] *  Environment: development
15:55:37 web.1    | [16202] *   Master PID: 16202
15:55:37 web.1    | [16202] *      Workers: 4
15:55:37 web.1    | [16202] *     Restarts: (✔) hot (✖) phased
15:55:37 web.1    | [16202] * Preloading application
15:55:37 web.1    | [16202] * Listening on http://127.0.0.1:3000
15:55:37 web.1    | [16202] * Listening on http://[::1]:3000
15:55:37 web.1    | [16202] Use Ctrl-C to stop
15:55:37 web.1    | [16202] - Worker 0 (PID: 16214) booted in 0.01s, phase: 0
15:55:37 web.1    | [16202] - Worker 1 (PID: 16215) booted in 0.01s, phase: 0
15:55:37 web.1    | [16202] - Worker 2 (PID: 16216) booted in 0.0s, phase: 0
15:55:37 web.1    | [16202] - Worker 3 (PID: 16217) booted in 0.0s, phase: 0
15:55:37 worker.1 | 2024-06-19T20:55:37.681Z pid=16203 tid=7nf INFO: Booted Rails 7.1.3.4 application in development environment
15:55:37 worker.1 | 2024-06-19T20:55:37.681Z pid=16203 tid=7nf INFO: Running in ruby 3.3.2 (2024-05-30 revision e5a195edf6) [arm64-darwin23]
15:55:37 worker.1 | 2024-06-19T20:55:37.681Z pid=16203 tid=7nf INFO: See LICENSE and the LGPL-3.0 for licensing details.
15:55:37 worker.1 | 2024-06-19T20:55:37.681Z pid=16203 tid=7nf INFO: Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org
15:55:37 worker.1 | 2024-06-19T20:55:37.681Z pid=16203 tid=7nf INFO: Sidekiq 7.2.4 connecting to Redis with options {:size=>10, :pool_name=>"internal", :url=>nil}
15:55:37 worker.1 | 2024-06-19T20:55:37.685Z pid=16203 tid=7nf INFO: Sidekiq 7.2.4 connecting to Redis with options {:size=>5, :pool_name=>"default", :url=>nil}

Is there anything else I can to do try to help debug this? Does anyone have any ideas about what's going on?

jagthedrummer commented 2 weeks ago

Just for the record I originally did all of the above from a terminal session running inside of tmux. I just tried with a plain session outside of tmux and I'm getting the same results.

Envek commented 2 weeks ago

Hey, thanks for reporting.

Unfortunately, I can't reproduce this behaviour. Can your local tmux has some custom configuration?

I wonder what may be different for the first and other processes as overmind uses a bit different commands to run the first process: https://github.com/DarthSim/overmind/blob/a9907243c989baac013c705bcec415f8b82cb8c8/start/tmux.go#L85 and second to last: https://github.com/DarthSim/overmind/blob/a9907243c989baac013c705bcec415f8b82cb8c8/start/tmux.go#L98

jagthedrummer commented 2 weeks ago

Thanks for responding @Envek. I do have a little bit of custom tmux configuration, but I don't think anything unusual. Is there anything in particular that you can think of that I should look for?

Here's my tmux config file if it helps: https://github.com/jagthedrummer/dotfiles/blob/master/tmux/tmux.conf

DarthSim commented 2 weeks ago

I guess, set-environment breaks the things since it rewrites PATH

jagthedrummer commented 2 weeks ago

Oooooh, good eye on that. I'll see if that makes a difference and report back.

Envek commented 1 week ago

But why set-environment PATH doesn't break the things for the first command? :thinking:

jagthedrummer commented 1 week ago

@DarthSim, good call on that set-environment call being in play. If I remove that from my .tmux.config file then overmind starts all of the processes in the Procfile successfully. Definitely seems odd that it doesn't cause problems for the first process type, but does cause problems for all others.

Envek commented 1 week ago

At the time being I would recommend to craft separate config for Overmind and use it instead (you can declare path to it in OVERMIND_TMUX_CONFIG env variable). See https://github.com/DarthSim/overmind?tab=readme-ov-file#specifying-tmux-config for details.