fukamachi / woo

A fast non-blocking HTTP server on top of libev
http://ultra.wikia.com/wiki/Woo_(kaiju)
MIT License
1.28k stars 98 forks source link

* Fix: don't fail when to many files are open #29

Closed kayhman closed 9 years ago

kayhman commented 9 years ago

Hi,

Just a quick fix to avoid to fail when too many sockets are already opened.

Regards,

Guillaume

fukamachi commented 9 years ago

I'm not sure if just ignoring the error is a good change. It's a serious error Woo's user should notice. The error will be disappeared if the server's max open file count is set properly.

eudoxia0 commented 9 years ago

The real fix is to increase the operating system's file handler limits in /etc/security/limits.conf and /etc/sysctl.conf.

kayhman commented 9 years ago

I don't think that changing the limit is a good option in my case. I can get a huge number of connections. I don't want to have to change this limit as my load increase on my servers.

What I want, is

Hence, I agree with fukamachi. I think that the error must be raised and catch later.

I've tried to do that, but I didn't find when and where "tcp-accept-cb" is called. I guess somewhere in libev ?

kayhman commented 9 years ago

In fact, with the current code, it seems to work as I expected, as my golang client failed with the message : "dial tcp 127.0.0.1:8080: too many open files"

kayhman commented 9 years ago

From the stack :

  0: (WOO.EV.TCP::TCP-ACCEPT-CB #<unavailable argument> #.(SB-SYS:INT-SAP #X7FFFDC011020) #<unavailable argument>)
  1: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/home/lapin/.cache/common-lisp/sbcl-1.2.11-linux-x64/1.2.11/home/lapin/Projects/woo/src/ev/socket.fasl") 70368614953312 703..
  2: ("foreign function: call_into_lisp")
  3: ("foreign function: funcall3")
  4: ("foreign function: callback_wrapper_trampoline")
  5: ("foreign function: #x20100E1C")
  6: ("foreign function: #x2009FF19A0")
  7: (WOO:RUN #<CLOSURE (LAMBDA (LACK.MIDDLEWARE.BACKTRACE::ENV) :IN "/home/lapin/.roswell/impls/ALL/ALL/quicklisp/dists/quicklisp/software/lack-20150709-git/src/middleware/backtrace.lisp") {10088C75DB}>..
  8: ((LAMBDA NIL :IN BORDEAUX-THREADS::BINDING-DEFAULT-SPECIALS))
  9: ((FLET #:WITHOUT-INTERRUPTS-BODY-1167 :IN SB-THREAD::INITIAL-THREAD-FUNCTION-TRAMPOLINE))
 10: ((FLET SB-THREAD::WITH-MUTEX-THUNK :IN SB-THREAD::INITIAL-THREAD-FUNCTION-TRAMPOLINE))
 11: ((FLET #:WITHOUT-INTERRUPTS-BODY-618 :IN SB-THREAD::CALL-WITH-MUTEX))
 12: (SB-THREAD::CALL-WITH-MUTEX #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK :IN SB-THREAD::INITIAL-THREAD-FUNCTION-TRAMPOLINE) {7FFFF0986C8B}> #<SB-THREAD:MUTEX "thread result lock" owner: #<SB-THREAD:THR..
 13: (SB-THREAD::INITIAL-THREAD-FUNCTION-TRAMPOLINE #<SB-THREAD:THREAD "clack-handler-woo" RUNNING {10088C7B23}> NIL #<CLOSURE (LAMBDA NIL :IN BORDEAUX-THREADS::BINDING-DEFAULT-SPECIALS) {10088C7AAB}> (#<S..
 14: ("foreign function: call_into_lisp")
 15: ("foreign function: new_thread_trampoline")

I can see that the TCP-ACCEPT-CB is started from woo:run. I've tried to put a restart inside with-event-loop, but It does not see my restart. (it is not listed by (compute-restart))

kayhman commented 9 years ago

Hi,

I've written a more satisfying fix, that still raises the error, but provides a restart in this case.

The restart can then be invoked in the woo::run function.

guillaume

fukamachi commented 9 years ago

I get your intention. I think that Woo should not stop when getting EMFILE too. It'll be fixed soon.

kayhman commented 9 years ago

Ok. Perfect. I'm curious to see how you'll solved this.

Thanks

kayhman commented 9 years ago

I realise that one of my commit was not online. The condition emfile-error is now raised when the error occurs.

fukamachi commented 9 years ago

Fixed by #32.