KnowSheet / Sherlock

Structured, append-only, immutable data persistence layer with publish-subscribe.
2 stars 1 forks source link

Fighting the deadlock. #32

Closed dkorolev closed 9 years ago

dkorolev commented 9 years ago

Fixed the deadlock.

Took me two and a half refactorings:

... after which the issue remained, the call to .join() in the destructor of ListenerThread (ex. ListenerScope) hangs.

After a couple more hours I concluded that there's a 99.99+% chance that deadlock happens when Notify(), that notifies that listener thread should shut down, as the listener scope object is being destructed, coincides with Wait() called from within the thread.

Subtle hint:

The effects of notify_one()/notify_all() and wait()/wait_for()/wait_until() take place in a single total order, so it's impossible for notify_one() to, for example, be delayed and unblock a thread that started waiting just after the call to notify_one() was made.

(via http://en.cppreference.com/w/cpp/thread/condition_variable/notify_one)

The solution in this pull request is to not rely on a single Notify(), followed by .join(), but to keep calling Notify(), if the thread did not confirm its acceptance within a short period of time. This required exposing WaitFor() (Wait() with an explicit upper bound on the amount of time to keep waiting) in WaitableAtomic<T>.

And this works now. @mzhurovich, please take a look!

mzhurovich commented 9 years ago

@dkorolev Not sure, I'm ready to check the logic now. So let's merge it and will take another look in a few days, as we discussed over the phone?

dkorolev commented 9 years ago

+1, you're the boss, boss :sunglasses: