derekkraan / horde

Horde is a distributed Supervisor and Registry backed by DeltaCrdt
MIT License
1.32k stars 106 forks source link

match error when stopping a process #241

Open flaviogrossi opened 2 years ago

flaviogrossi commented 2 years ago

I'm not able to reproduce this, but we captured this error after terminating a process under a dynamic supervisor.

This happened after replying with {:stop, :normal, ...} to a genserver call in a supervised process. The process has terminated correctly, but its supervisor has crashed with the following trace:

GenServer :\"Presence.User.UserSupervisor.3.HordeSupervisor\" terminating
** (MatchError) no match of right hand side value: {nil, #Reference<0.1078700752.1562771457.143737>}
   (horde 0.8.4) lib/horde/dynamic_supervisor_impl.ex:233: Horde.DynamicSupervisorImpl.handle_cast/2
   (stdlib 3.15.2) gen_server.erl:695: :gen_server.try_dispatch/4
   (stdlib 3.15.2) gen_server.erl:771: :gen_server.handle_msg/6
   (stdlib 3.15.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: {:\"$gen_cast\", {:disown_child_process, 65042756075423944957220388867353681208}}" tags=default pid=<0.3322.0> module=gen_server function=error_info/7 file=gen_server.erl line=949 vm_pid=2000 host=vm1-3vqpiro-40 erl_level=error crash_reason="{%MatchError{term: {nil, #Reference<0.1078700752.1562771457.143737>}}, [{Horde.DynamicSupervisorImpl, :handle_cast, 2, [file: 'lib/horde/dynamic_supervisor_impl.ex', line: 233]}, {:gen_server, :try_dispatch, 4, [file: 'gen_server.erl', line: 695]}, {:gen_server, :handle_msg, 6, [file: 'gen_server.erl', line: 771]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}" domain=[:otp] error_logger="%{report_cb: &:gen_server.format_log/1, tag: :error}" gl=<0.3029.0> mfa="{:gen_server, :error_info, 7}" registered_name=Presence.User.UserSupervisor.3.HordeSupervisor report_cb=&:gen_server.format_log/2

GenServer #PID<0.3325.0> terminating
** (stop) exited in: GenServer.call(:\"Presence.User.UserSupervisor.3.HordeSupervisor\", :horde_shutting_down, 5000)
   ** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
   (elixir 1.11.4) lib/gen_server.ex:1017: GenServer.call/3
   (horde 0.8.4) lib/horde/signal_shutdown.ex:21: anonymous fn/1 in Horde.SignalShutdown.terminate/2
   (elixir 1.11.4) lib/enum.ex:798: Enum.\"-each/2-lists^foreach/1-0-\"/2
   (stdlib 3.15.2) gen_server.erl:733: :gen_server.try_terminate/3
   (stdlib 3.15.2) gen_server.erl:918: :gen_server.terminate/10
   (stdlib 3.15.2) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: {:EXIT, #PID<0.3319.0>, :shutdown}" tags=default pid=<0.3325.0> module=gen_server function=error_info/7 file=gen_server.erl line=949 vm_pid=2000 host=vm1-3vqpiro-40 erl_level=error crash_reason="{{:noproc, {GenServer, :call, [:\"Presence.User.UserSupervisor.3.HordeSupervisor\", :horde_shutting_down, 5000]}}, [{GenServer, :call, 3, [file: 'lib/gen_server.ex', line: 1017]}, {Horde.SignalShutdown, :\"-terminate/2-fun-0-\", 1, [file: 'lib/horde/signal_shutdown.ex', line: 21]}, {Enum, :\"-each/2-lists^foreach/1-0-\", 2, [file: 'lib/enum.ex', line: 798]}, {:gen_server, :try_terminate, 3, [file: 'gen_server.erl', line: 733]}, {:gen_server, :terminate, 10, [file: 'gen_server.erl', line: 918]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}" domain=[:otp] error_logger="%{report_cb: &:gen_server.format_log/1, tag: :error}" gl=<0.3029.0> mfa="{:gen_server, :error_info, 7}" report_cb=&:gen_server.format_log/2

it seems that here the case when pop_items returns nil is not handled.

Please let me know if more details are needed, we are currently trying to better understand what caused this chain of events, in particular if it is expected for pop_items to return nil in that case

Frost commented 2 years ago

I'm also seeing this behaviour. Let me know if there's anything I can help with.