search

airbnb / nerve

A service registration daemon that performs health checks; companion to airbnb/synapse
MIT License
942 stars 151 forks source link

Fix the ZK thread leak while closing connection. #67

Closed amjith closed 9 years ago

amjith commented 9 years ago

Zookeeper client spawns a reconnect thread that is not killed when zk.close is called, but it is only killed when zk.close! (https://github.com/zk-ruby/zk/blob/master/lib/zk/client/threaded.rb#L332) is called.

In order to trigger the thread leak, have nerve running with a few services, watch the number of threads for the nerve process, then kill zookeeper instances (wait for 30secs timeout for ZK ping). Then relaunch ZK and watch the thread count. It will grow by 2 per process.

This fixes the thread leak by joining the reconnect thread when Zookeeper goes down.

amjith commented 9 years ago

I'm not really sure why Travis failed with bundle exec rake exited with 1. I haven't made any changes that could affect rake.

amjith commented 9 years ago

Fixed the test. Travis is passing now.

igor47 commented 9 years ago

great, thanks for the PR!