celluloid / reel

UNMAINTAINED: See celluloid/celluloid#779 - Celluloid::IO-powered web server
https://celluloid.io
MIT License
596 stars 87 forks source link

HAProxy crashing Reel::Server #122

Closed koaps closed 5 years ago

koaps commented 10 years ago

Hi all,

HAProxy HTTP checks keeps crashing the Reel::Server.

I'm seeing this when running the spy_hello_world.rb

*\ Starting server on http://127.0.0.1:8001 +++ 127.0.0.1:45255 (AF_INET) connected GET / HTTP/1.1 Host: webapp

HTTP/1.1 200 OK Connection: Keep-Alive Content-Length: 13

hello, world!--- 127.0.0.1:45255 (AF_INET) disconnected E, [2013-11-15T00:20:05.154000 #40167] ERROR -- : Reel::Server crashed! Reel::StateError: already processing a request /home/koaps/jruby/lib/ruby/gems/shared/gems/reel-0.5.0.pre/lib/reel/connection.rb:55:in request' /home/koaps/jruby/lib/ruby/gems/shared/gems/reel-0.5.0.pre/lib/reel/connection.rb:78:ineach_request' hello_world.rb:13:in (root)' org/jruby/RubyProc.java:276:incall' /home/koaps/jruby/lib/ruby/gems/shared/gems/reel-0.5.0.pre/lib/reel/server.rb:52:in handle_connection' org/jruby/RubyKernel.java:1683:inpublic_send' /home/koaps/jruby/lib/ruby/gems/shared/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in dispatch' /home/koaps/jruby/lib/ruby/gems/shared/gems/celluloid-0.15.2/lib/celluloid/calls.rb:122:indispatch' /home/koaps/jruby/lib/ruby/gems/shared/gems/celluloid-0.15.2/lib/celluloid/actor.rb:322:in handle_message' /home/koaps/jruby/lib/ruby/gems/shared/gems/celluloid-0.15.2/lib/celluloid/actor.rb:416:intask' /home/koaps/jruby/lib/ruby/gems/shared/gems/celluloid-0.15.2/lib/celluloid/tasks.rb:55:in initialize' /home/koaps/jruby/lib/ruby/gems/shared/gems/celluloid-0.15.2/lib/celluloid/tasks/task_fiber.rb:13:increate' W, [2013-11-15T00:20:05.158000 #40167] WARN -- : Terminating task: type=:call, meta={:method_name=>:run}, status=:iowait

If I start the server on another port that HAProxy isn't checking and hit it with links, there's no issues.

*\ Starting server on http://127.0.0.1:8002 +++ 127.0.0.1:44958 (AFINET) connected GET / HTTP/1.1 Host: 127.0.0.1:8002 User-Agent: ELinks/0.12pre6 (textmode; SunOS 5.11 i86pc; 213x68-2) Accept: /_ Accept-Language: en Connection: Keep-Alive

HTTP/1.1 200 OK Connection: Keep-Alive Content-Length: 13

hello, world!--- 127.0.0.1:44958 (AF_INET) disconnected

I had simular issues with Reel-Rack and I was able to get it to stop crashing by setting HAProxy httpchk to use HTTP/1.1 but the Reel::Server crashes with the same config.

From HAProxy config file:

option httpchk GET / HTTP/1.1\r\nHost:\ webapp

I'm assuming HAProxy discounts right away and Reel::Server doesn't like that, but I am not sure.

tarcieri commented 10 years ago

It looks like there's a potential problem with the request not getting cleared from the connection if it snaps shut while writing the response. This could use some better checking of the request state machine.

sax commented 10 years ago

HAProxy aborts the tcp connection after the first ack on a status check.

damned commented 8 years ago

i found this issue while raising #229 - i think possibly related