Connection overload when using futuroscope

[alxgsv]

Hi guys. I tried futuroscope and it's great! Great idea and simple design.

But when I use it in Rails application workers it generates too many connections to database. Is there any way to utilize existing ActiveRecord connections?

Sorry if I missed something.

[josepjaume]

Hi @LongMan !

First of all thanks for trying out futuroscope, hope you liked it :)

Unfortunately I think that having a connection per thread is intrinsic on Rails. Check out the docs:

http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/ConnectionPool.html

The only thing you can do to keep them under control is either limit futuroscope's workers (https://github.com/codegram/futuroscope#worker-pool) or inside your database.yml settings: http://stackoverflow.com/questions/15086880/correct-setting-of-database-connection-pool-database-yml-for-single-threaded-rai

Does that solve your problem?

[alxgsv]

Thanks for such a fast and wide answer!

Problem not solved, but now I know nothing missed :-) Will think about it.

[josepjaume]

Hey! I actually found this in the documentation: http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/ConnectionPool.html#method-i-with_connection

So each future should run inside a with_connection block. I may provide a futuroscope-activerecord at some point to deal with that. Could you tell me what happens if you surround your block like this?

something = future do
  ActiveRecord::Base.connection_pool.with_connection{ [....] } 
end

[alxgsv]

Yeah, I read http://bibwild.wordpress.com/2011/11/14/multi-threading-in-rails-activerecord-3-0-3-1/ about that. Will try and report here.

[alxgsv]

Result: https://www.evernote.com/shard/s2/sh/eb68d620-d46d-4628-ac1f-0424a6351b8a/01ff611fc03fc2f50a8ddafb969e53cf

I see 1.75x grow in connections. But this app is not only one that uses this mysql server. So I assume that there is 2x grow and it's working like this:

When worker gets into #with_connection it creates new connection and reuse it when it gets into #with_connection again. Maybe I am wrong but seems like that.

Bad news is that 2x is still big. Good news is that it grows only upto 2x, unlinke without with_connection

[josepjaume]

This 2x increment is because you'll have at least one extra thread using activerecord when using values. In order to use futuroscope efficiently, you'll have to increase you mysql pool in database.yml up to as much parallelism you want + 1.