randomPoison / thespian

An experiment in designing an ergonomic actor framework for Rust
1 stars 0 forks source link

No way to prevent deadlocks with 2-way actor communication #9

Open randomPoison opened 4 years ago

randomPoison commented 4 years ago

It's currently very easy to deadlock two actors that send each other messages. For example:

#[derive(Actor)]
pub struct Foo {
    bar: BarProxy,
}

#[thespian::actor]
impl Foo {
    pub fn tell_bar(&mut self) {
        self.bar.tell_foo().await;
    }

    pub fn tell_me(&self) {}
}

#[derive(Actor)]
pub struct Bar {
    foo: FooProxy,
}

#[thespian::actor]
impl Bar {
    pub fn tell_foo(&mut self) {
        self.foo.tell_me().await.
    }
}

In this case when Bar::tell_foo calls Foo::tell_me, both actors will deadlock. This is is because the Foo actor is waiting for Foo::tell_bar to complete before it can process another message, but Foo::tell_bar is waiting on Foo::tell_me to complete. Since tell_me won't be processed until tell_bar finishes, neither actor can progress and so both deadlock.

In this specific case, the issue is that we have no way to enqueue a message for an actor without also waiting for the actor to respond to this message. If we had a "fire and forget" option for enqueuing a message without waiting for the response, tell_foo could send the tell_me message to Foo without waiting for the message to be processed, thereby avoiding the deadlock.

However, at a more fundamental level, the fact that messages can return results will always introduce the possibility of a deadlock. The only way to fully prevent deadlocks would be to have enqueuing a message always be a synchronous action that either succeeds immediately or fails due to the message queue being full (or the actor having stopped).

I'd like to make thespian as deadlock-free as possible. It may not be possible to completely prevent all deadlocks, but the API should push users to usage patterns that avoid deadlocks by default. I'm going to have to do some research into how deadlocks are handled in other actor frameworks like Actix and what mechanisms can be used to prevent/avoid them.

randomPoison commented 4 years ago

As of #11 being merged, it's now possible to send a message without awaiting a response. This makes it possible to at least fix a deadlock once discovered, though deadlocks are still possible to cause in the manner described above.