Service with refresh function called by thread hangs on occasion #144

Closed zeenix closed 1 year ago

In GitLab by @tmuehlbacher on Feb 2, 2021, 12:33

I have a service that needs to poll values in the background and therefore cannot wait for the next event to come in from a client. So I created another thread that calls a refresh function once a second, similar to this using zbus 2.0.0-beta:

use std::convert::TryInto;
use std::error::Error;
use zbus::{dbus_interface, fdo};

struct Greeter {
    count: u64,
}

#[dbus_interface(name = "org.zbus.MyGreeter1")]
impl Greeter {
    fn refresh(&mut self) {
        self.count += 1;
    }

    #[dbus_interface(property)]
    fn count(&self) -> u64 {
        self.count
    }
}

fn main() -> Result<(), Box<dyn Error>> {
    let connection = zbus::Connection::new_session()?;
    fdo::DBusProxy::new(&connection)?.request_name(
        "org.zbus.MyGreeter",
        fdo::RequestNameFlags::ReplaceExisting.into(),
    )?;

    let mut object_server = zbus::ObjectServer::new(&connection);
    let greeter = Greeter { count: 0 };
    object_server.at(&"/org/zbus/MyGreeter".try_into()?, greeter)?;

    let _t = std::thread::spawn(move || loop {
        std::thread::sleep(std::time::Duration::from_secs(1));
        connection
            .call_method(
                Some("org.zbus.MyGreeter"),
                "/org/zbus/MyGreeter",
                Some("org.zbus.MyGreeter1"),
                "Refresh",
                &(),
            )
            .unwrap();
    });

    loop {
        if let Err(err) = object_server.try_handle_next() {
            eprintln!("{}", err);
        }
    }
}

To get the hangs to happen (or to make them noticeable?), run for example:

$ for i in $(seq 1000); do busctl --user get-property org.zbus.MyGreeter /org/zbus/MyGreeter org.zbus.MyGreeter1 Count; done
...
t 12
t 12
t 12
t 12
t 12
Failed to get property Count on interface org.zbus.MyGreeter1: Connection timed out
t 13
t 13
t 13
t 13
t 13
t 13
t 13
...

I could not get this to happen if I comment out the thread in the main function.

@tmuehlbacher seems like it's not a complete hang, since client is subsequently able to make successful calls? An easy workaround would be to use separate connections for server and client-side.

In GitLab by @tmuehlbacher on Feb 2, 2021, 14:09

Yeah, ok cloning the connection and moving the cloned one into the thread doesn't work but creating a separate connection2 (e.g.) just like the first one does appear to fix the problem.

Shouldn't clone() also work?

Shouldn't clone() also work?

No, cause it's the same underlying connection. :)

In GitLab by @tmuehlbacher on Feb 2, 2021, 15:39

Okay :)

Well anyway looking at output of the busctl calls it looks like the client doesn't get a reply if it calls right when the refresh is happening. Like that call is basically swallowed by the server.

With the workaround of a separate connection (thanks :) it becomes less pressing, so perhaps I'll look into it when I have time at some point. Or do you think it's not actually possible to do it with just one connection?

Well anyway looking at output of the busctl calls it looks like the client doesn't get a reply if it calls right when the refresh is happening. Like that call is basically swallowed by the server.

Actually that's very much expected as ObjectServer::try_handle_next() just takes any next message on the connection. So the solution here would be for it to use the (relatively) new Connection::receive_specific instead to only take incoming method calls from the connection.

With the workaround of a separate connection (thanks :) it becomes less pressing, so perhaps I'll look into it when I have time at some point. Or do you think it's not actually possible to do it with just one connection?

Yeah, if the above solution works (as it really should), then yeah. Another workaround involving just on connection would be for you to manually receive messages from the connection and then call ObjectServer::dispatch_message.

mentioned in commit 69f20afccd0547e8dd5d5541cfdadcaa9b247298

mentioned in commit cf79cb6f7006fb1fc72b3b3f6302d9aa92266139

mentioned in commit 60f8a8188a4d30284ee160cc3bf361f19584e777

mentioned in commit 3a2a37f613fce67707853867c222e477694382d1

dbus2 / zbus-old

Service with refresh function called by thread hangs on occasion #144