Socket closed remotely causes a match error in `:ldclient_event_dispatch_httpc`

quinlanjager commented 4 years ago

Describe the bug

When sending events to LaunchDarkly through the erlang SDK we occasionally see the GenServer crashing with the following error (This an Elixir application).

GenServer :ldclient_event_process_server_default terminating
** (MatchError) no match of right hand side value: {:error, :socket_closed_remotely}
    (ldclient) /tmp/build_dc5d350dd48fc90045058a6929a019cf/deps/ldclient/src/ldclient_event_dispatch_httpc.erl:29: :ldclient_event_dispatch_httpc.send/4
    (ldclient) /tmp/build_dc5d350dd48fc90045058a6929a019cf/deps/ldclient/src/ldclient_event_process_server.erl:97: :ldclient_event_process_server.handle_cast/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

To reproduce This is tough to reproduce exactly, but I think sending event requests to LaunchDarkly while the socket is closed will do the trick.

Expected behavior A clear and concise description of what you expected to happen.

Perhaps this should return an {:error, permanent or temporary, ...} tuple as is appropriate.

Logs If applicable, add any log output related to your problem.

SDK version 1.0.0-beta2

Language version, developer tools Elixir 1.9.4, Erlang/OTP 21

hez commented 4 years ago

We have seen this as well.... It does restart almost immediately though so not sure if it is a problem.

[info] Starting event processor for :app with name :ldclient_event_process_server_app

bwoskow-ld commented 4 years ago

Thanks for filing. Based on your description it does sound like this issue might be happening, as you surmised, when the event stream disconnects and reconnects to LaunchDarkly. This most often happens when we redeploy our service however it can also happen due to any sort of networking blip between your services and ours.

We'll take a look into this prior to our GA release.

hez commented 4 years ago

Looks like there is a fix in the up coming release, thanks!

re: https://github.com/launchdarkly/erlang-server-sdk/pull/27

zurab-darkly commented 4 years ago

@hez that is correct. 1.0.0-beta4 is now available containing the fix for this bug.

launchdarkly / erlang-server-sdk

Socket closed remotely causes a match error in `:ldclient_event_dispatch_httpc` #23