zombocom / puma_worker_killer

Automatically restart Puma cluster workers based on max RAM available
747 stars 77 forks source link

Not seeing 'PumaWorkerKiller: Consuming XXX mb' stdout logging after SIGUSR1 signal to master pid #99

Closed dfuentes77 closed 2 years ago

dfuentes77 commented 2 years ago

This is what I have in puma config file

workers Etc.nprocessors
preload_app! false
prune_bundler
wait_for_less_busy_worker 
fork_worker
plugin :tmp_restart

extra_runtime_dependencies ['puma_worker_killer']

on_worker_fork do
  require 'puma_worker_killer'

  PumaWorkerKiller.config do |config|
    config.ram           = 2048 # mb
    config.frequency     = 10 # seconds
  end

  PumaWorkerKiller.start
end

before_fork do
    require 'puma_worker_killer'

    PumaWorkerKiller.config do |config|
      config.ram           = 2048 # mb
      config.frequency     = 10 # seconds
    end

    PumaWorkerKiller.start
  end

Basically after I do a phased-restart (SIGUSR1), the puma killer stops logging the consumed memory.

dfuentes77 commented 2 years ago

referring to puma ticket as well since I'm not sure if this is just an incompatibility between puma killer and the new fork_worker: https://github.com/puma/puma/issues/2713

dfuentes77 commented 2 years ago

As a separate issue (I think) but related since it happens every time I start puma or do a phased-restart on macos, I get the following type errors:

#<Thread:0x00007fcdc70d9c00@<path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/auto_reap.rb:14 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
    13: from <path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/auto_reap.rb:17:in `block in start'
    12: from <path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/reaper.rb:19:in `reap'
    11: from <path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/puma_memory.rb:25:in `workers_stopped?'
    10: from <path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/puma_memory.rb:29:in `running?'
     9: from <path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/puma_memory.rb:61:in `workers'
     8: from <path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/puma_memory.rb:74:in `set_workers'
     7: from <path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/puma_memory.rb:74:in `each'
     6: from <path to puma_worker_killer-0.3.1>/lib/puma_worker_killer/puma_memory.rb:75:in `block in set_workers'
     5: from <path to get_process_mem-0.2.7>/lib/get_process_mem.rb:69:in `mb'
     4: from <path to get_process_mem-0.2.7>/lib/get_process_mem.rb:61:in `bytes'
     3: from <path to get_process_mem-0.2.7>/lib/get_process_mem.rb:122:in `darwin_memory'
     2: from <path to get_process_mem-0.2.7>/lib/get_process_mem/darwin.rb:47:in `resident_size'
     1: from <path to get_process_mem-0.2.7>/lib/get_process_mem/darwin.rb:54:in `get_proc_pidinfo'
<path to get_process_mem-0.2.7>/lib/get_process_mem/darwin.rb:54:in `proc_pidinfo': no implicit conversion from nil to integer (TypeError)
schneems commented 2 years ago

To make any progress we'll need to reproduce the issue. Can you provide an example app? https://www.codetriage.com/example_app

dfuentes77 commented 2 years ago

I'm closing since fork_worker is still experimental, looks like it's going to eventually change, and has other issues that prevents me from using it in production. I'll be waiting until it's more stable.