search

celluloid / celluloid-io

UNMAINTAINED: See celluloid/celluloid#779 - Evented sockets for Celluloid actors
https://celluloid.io
MIT License
879 stars 93 forks source link

Errno::EMFILE: Too many open files - pipe #123

Closed blanchma closed 9 years ago

blanchma commented 9 years ago

Case: Running several times a class which do http requests:

class SmsService include Celluloid::IO

def initialize ... end

def deliver response = HaHelper::HTTP.get(url, ssl_socket_class: Celluloid::IO::SSLSocket).to_s end end

Environment: rvm use 1.9.3-p484

celluloid (0.16.0)
celluloid-io (0.16.1)
http (0.6.2)

Stacktrace:

Celluloid::PoolManager: async call perform aborted! Errno::EMFILE: Too many open files - pipe /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-io-0.16.1/lib/celluloid/io/reactor.rb:16:in new' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-io-0.16.1/lib/celluloid/io/reactor.rb:16:ininitialize' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-0.16.0/lib/celluloid/evented_mailbox.rb:9:in new' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-0.16.0/lib/celluloid/evented_mailbox.rb:9:ininitialize' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-io-0.16.1/lib/celluloid/io/mailbox.rb:6:in initialize' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-0.16.0/lib/celluloid/actor.rb:105:innew' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-0.16.0/lib/celluloid/actor.rb:105:in initialize' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-0.16.0/lib/celluloid/cell.rb:21:innew' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-0.16.0/lib/celluloid/cell.rb:21:in initialize' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-0.16.0/lib/celluloid.rb:168:innew' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/celluloid-0.16.0/lib/celluloid.rb:168:in new' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/http-0.6.2/lib/http/chainable.rb:140:inbranch' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/http-0.6.2/lib/http/chainable.rb:52:in request' /Users/blanchma/.rvm/gems/ruby-1.9.3-p484@applicationservice/gems/http-0.6.2/lib/http/chainable.rb:12:inget' /Users/blanchma/workspace/applicationservice/app/services/sms_service.rb:49:in `deliver'

ioquatix commented 9 years ago

Are you running this on Mac OS X? It has an absurdly low file limit.

blanchma commented 9 years ago

Yes, I am. I read the same about the mac. My concern is that even in Linux this could happen because nothing handle the amount of open pipes.

tarcieri commented 9 years ago

@blanchma from a cursory glance it looks like you're probably making an unbounded number of requests and exhausting available file descriptors. Can you set some sort of cap on the number of outstanding requests you make?

ioquatix commented 9 years ago

It's simple really, if you try to have more file descriptors than the operating system allows, there will be an error.

File Handle Limitations On some platforms (e.g. Mac OS X) the number of file descriptors is relatively low by default and should be increased by calling ulimit -n 10000 before running tests or even before starting a server which expects a large number of concurrent incoming connections.

digitalextremist commented 9 years ago

As per responses, this is a common issue on an Operating System level -- not Celluloid::IO. @ioquatix shows you the solution using ulimit under Mac OS X.