camper_van goes berserk when trying to reconnect

[yorickpeterse]

At times it appears that camper_van is no longer able to connect to the Campfire servers (not exactly sure why/when this happens) which will result in camper_van constantly trying to reconnect. This introduces two problems:

• When logging output to a file (even when the log level is set to "info") this will very quickly result in a very large log file. In my case I ended up with a 1,2GB (largest so far was 1,5GB) log file in a matter of minutes.
• Both CPU usage and memory usage skyrockets up to the point where Monit started complaining that around 80% of my server's memory (which is around 1,6GB) was being used. CPU usage in this case sits at around 100%.

I'm not entirely sure what's causing it as it appears to be random, though I've noticed an increase of this behaviour in the past 2-3 weeks.

I suspect part of the problem can be resolved by only reconnecting every N seconds (e.g. every 10 seconds) and increasing the time with every failed attempt. However, I can't really figure out where the reconnecting related code is located.

[blalor]

I've seen some weirdness around this, too. I have a theory: when CV has to reconnect, it eventually ends up with multiple simultaneous connections. This causes messages to get posted to IRC with as many duplicates as there are connections.

Yeow, just confirmed that. camper_van's pid is 14646; netstat -anp | grep 14646 shows 565 (!) established connections:

tcp        0      0 108.59.4.79:48867           204.62.114.22:443           ESTABLISHED 14646/ruby          

netstat -anp | grep 14646 | grep 'ESTABLISHED 14646' | wc -l
565

[zerowidth]

Ouch. This doesn't really surprise me, but it sure sucks. The reconnect code is kinda hidden/implicit in the firering library, but it's not clear if it cleans up after itself or not. Seems the evidence points toward not.

Do you have any suggestions? At some point this is going to start sucking enough that I rewrite the thing in celluloid...

[blalor]

I'm afraid I don't have any suggestions I can make; I'm not a Rubyist and don't have any experience with firering…

[yorickpeterse]

If it's hidden in Firering then you're really left with two options: fix Firering or get rid of it. Since Firering already uses a heavily outdated EventMachine version I don't think the latter would be such a bad idea, it would be time consuming though.

[blalor]

Would another alternative be to just disable reconnects? It'd be easier for me to deal with restarting the process ( while :; do camper_van ; done) than to have cope with 500+ simultaneous connections. ;-)

[rwilcox]

I saw this reconnect error when using Adium's built in IRC client, but when I connected with Textual everything seems to be working fine.

Maybe this information is helpful, maybe not.

[lunks]

I'll close this as stale, let me know if there is anything to be done. IMHO this should be reported on firering and fix it there.