Fix Kontena::Observable concurrency

[SpComb]

The Kontena::Observable class is used to share the continuously updated Docker::State between multiple service actors, using a single Docker::Actor to maintain the Docker::State across crashes/restarts of both the Docker and Service actor crashes.

This means that the observable state must be concurrency-safe across multiple actors. The current implementation uses a Celluloid::Condition variable to provide a neat synchronous API:

  def start
    self.async.run
    self.async.other_backgroud_tasks
  end

  def run
    @observable.observe do |state|
      self.apply(state)
    end
  end

Unfortunately the Celluloid::Condition variable lacks integration with any kind of mutex/semaphore, and that leads to race conditions typically solved using a combined mutex + condition variable approach: Ruby core ConditionVariable

The correct approach would probably be to make each Observable a separate Actor responsible for synchronizing the shared state. The question is, how to implement the current API, yielding to a receiver block on updates.

[SpComb]

The Condition race may have been fixed in 2ce7537 by getting rid of the unorthodox ruby Mutex, and using sync Actor methods for the operations instead.

Still worth figuring out how the Celluloid mailbox operations behave, if there's some more efficient way to implement the same primitive.

[SpComb]

The current Observable API seems to behave correctly as designed now.