mtrudel / bandit

Bandit is a pure Elixir HTTP server for Plug & WebSock applications
MIT License
1.69k stars 82 forks source link

Bandit from 1.3 to 1.5 ENOTCONN errors #371

Closed mjquinlan2000 closed 5 months ago

mjquinlan2000 commented 5 months ago

Hello,

I recently upgraded some production servers from bandit 1.3 to 1.5.4 and I'm seeing this error pop up in my APM logs:

[error] ** (RuntimeError) Unable to obtain transport_info: :enotconn
    (bandit 1.5.4) lib/bandit/transport_info.ex:26: Bandit.TransportInfo.init/1
    (bandit 1.5.4) lib/bandit/pipeline.ex:67: Bandit.Pipeline.build_conn!/5
    (bandit 1.5.4) lib/bandit/pipeline.ex:31: Bandit.Pipeline.run/4
    (bandit 1.5.4) lib/bandit/http1/handler.ex:12: Bandit.HTTP1.Handler.handle_data/3
    (bandit 1.5.4) lib/bandit/delegating_handler.ex:18: Bandit.DelegatingHandler.handle_data/3
    (bandit 1.5.4) lib/thousand_island/handler.ex:379: Bandit.DelegatingHandler.handle_info/2
    (stdlib 5.2.3) gen_server.erl:1095: :gen_server.try_handle_info/3
    (stdlib 5.2.3) gen_server.erl:1183: :gen_server.handle_msg/6
    (stdlib 5.2.3) proc_lib.erl:241: :proc_lib.init_p_do_apply/3

I have walked the source code and noticed that you are now raising these errors in transport_info.ex. To be honest I've never happened across ENOTCONN before and I'm not really sure what's causing it. Any help pointing me in the right direction would be greatly appreciated.

Elixir Version Info:

Erlang/OTP 26 [erts-14.2.5] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

Elixir 1.16.2 (compiled with Erlang/OTP 26)

Thank you

mtrudel commented 5 months ago

This happens when the client closes the connection early, before issuing a request and before the server even has time to obtain info about the connection (in this case, the server is attempting to obtain info about the remote client's host and port, but the connection has already been closed). It's a benign condition and just 'one of those things' that's part and parcel to running an internet connected service.

I've got a bit of work in the next few days to make these raise as any other prorocol-facing error does (ie: in one line :short form by default).

mjquinlan2000 commented 5 months ago

@mtrudel thank you for the quick response. Can you clarify this:

I've got a bit of work in the next few days to make these raise as any other prorocol-facing error does (ie: in one line :short form by default).

The problem that we're seeing with it logging to our APM solution (Sentry) is that it happens so often that it's eating up our plan's error allotment. The Sentry library wraps the Plug request and reports all raised errors so we would prefer it if Bandit didn't call raise/2 at all in these scenarios that aren't really a problem.

In the meantime, I've downgraded our library back to ~> 1.3.0

Thank you

mtrudel commented 5 months ago

It'll continue to raise; being noisy on errors is one of Bandit's core values. The fix here is to teach Sentry about bandit errors, similar to how it knows about cowboy errors and chooses not to report them. There's a discussion of this in the #bandit Slack channel at the moment, and the requisite changes will also be coming out in the next few days.

tl;dr you should be fine to update to the next release (along with a sentry library change / config change) and not bankrupt yourself on allotments.

mjquinlan2000 commented 5 months ago

Thank you so much! I've caught up on the Slack channel. I'll look out for the next release

mtrudel commented 4 months ago

1.5.5 just released with these changes, also filed https://github.com/getsentry/sentry-elixir/pull/739 upstream to get sentry to add :bandit to the list of ignored domains (in the interim they provide a config point for this if you so desire).