How to use signals

[awarecan]

It looks like when you processing interface, you ignored signal at all. https://github.com/facebookincubator/pystemd/blob/628655ae61dff66b39feed52c89e0bfda00c4f38/pystemd/base.py#L199-L219

So, is there any support for signal now? Any plan to add the support? I am looking for use your library, but I would not use it without support of signal that is very important for my project (current base on pydbus, but I would prefer a library doesn't depends on python-gi)

[aleivag]

we have no plan to implement signals, because so far, we have not need it, but if you have a use case, and you would like to share it with us (tell us why you need it, and how you will use them), i'll be happy to look into it and try to support for it.

(also patches are always welcome) :D

[awarecan]

My application is super simple, just a systemd services state monitor. User can set which services they want to monitor, and get "real time" notification when the service state changed.

[aleivag]

@awarecan cool, so i think you can get that now with pystemd in the current state, i actually do this in pystemd.run if you specify the option wait=True, what i do on that function, is start a transient unit, and then open a bus and subscribe to PropertiesChange dbusevent on that unit

you can check it out in:

https://github.com/facebookincubator/pystemd/blob/master/pystemd/run.py#L240-L244

but a quick example (that probably i should eventually add to the example folder could be):

Assuming that a unit name my_custom_sleep.service exists you can

import select

from pystemd.systemd1 import Unit
from pystemd.dbuslib import DBus
from pystemd.DBus import Manager as DBusManager

unit = Unit("my_custom_sleep.service")
mstr = (
    (
        "type='signal',"
        "sender='org.freedesktop.systemd1',"
        "path='{}',"
        "interface='org.freedesktop.DBus.Properties',"
        "member='PropertiesChanged'"
    )
    .format(unit.path.decode())
    .encode()
)

with DBus() as monbus, DBusManager(bus=monbus) as man:
    man.Monitoring.BecomeMonitor([mstr], 0)
    fd = monbus.get_fd()
    while True:
        select.select([fd], [], [])
        m = monbus.process()
        if m.is_empty():
            continue

        m.process_reply(False)
        print(m.body)
        if m.get_path() == unit.path:
            if m.body[1].get(b"SubState") in (b"exited", b"failed", b"dead"):
                break

and this will dump this lines, each time that one of the properties that emit change... well change...

[b'org.freedesktop.systemd1.Service', {b'MainPID': 3776, b'ControlPID': 0, b'StatusText': b'', b'StatusErrno': 0, b'Result': b'success', b'USBFunctionDescriptors': b'', b'USBFunctionStrings': b'', b'UID': 4294967295, b'GID': 4294967295, b'NRestarts': 0, b'ExecMainStartTimestamp': 1529038632232687, b'ExecMainStartTimestampMonotonic': 2466232687, b'ExecMainExitTimestamp': 0, b'ExecMainExitTimestampMonotonic': 0, b'ExecMainPID': 3776, b'ExecMainCode': 0, b'ExecMainStatus': 0}, [b'ExecStartPre', b'ExecStart', b'ExecStartPost', b'ExecReload', b'ExecStop', b'ExecStopPost']]

[b'org.freedesktop.systemd1.Unit', {b'ActiveState': b'active', b'SubState': b'running', b'StateChangeTimestamp': 1529038632232879, b'StateChangeTimestampMonotonic': 2466232879, b'InactiveExitTimestamp': 1529038632232879, b'InactiveExitTimestampMonotonic': 2466232879, b'ActiveEnterTimestamp': 1529038632232879, b'ActiveEnterTimestampMonotonic': 2466232879, b'ActiveExitTimestamp': 0, b'ActiveExitTimestampMonotonic': 0, b'InactiveEnterTimestamp': 0, b'InactiveEnterTimestampMonotonic': 0, b'Job': (0, b'/'), b'ConditionResult': True, b'AssertResult': True, b'ConditionTimestamp': 1529038632228522, b'ConditionTimestampMonotonic': 2466228523, b'AssertTimestamp': 1529038632228524, b'AssertTimestampMonotonic': 2466228524}, []]

[b'org.freedesktop.systemd1.Service', {b'MainPID': 0, b'ControlPID': 0, b'StatusText': b'', b'StatusErrno': 0, b'Result': b'success', b'USBFunctionDescriptors': b'', b'USBFunctionStrings': b'', b'UID': 4294967295, b'GID': 4294967295, b'NRestarts': 0, b'ExecMainStartTimestamp': 1529038632232687, b'ExecMainStartTimestampMonotonic': 2466232687, b'ExecMainExitTimestamp': 1529038692242131, b'ExecMainExitTimestampMonotonic': 2526242131, b'ExecMainPID': 3776, b'ExecMainCode': 1, b'ExecMainStatus': 0}, [b'ExecStartPre', b'ExecStart', b'ExecStartPost', b'ExecReload', b'ExecStop', b'ExecStopPost']]

[b'org.freedesktop.systemd1.Unit', {b'ActiveState': b'inactive', b'SubState': b'dead', b'StateChangeTimestamp': 1529038692242636, b'StateChangeTimestampMonotonic': 2526242636, b'InactiveExitTimestamp': 1529038632232879, b'InactiveExitTimestampMonotonic': 2466232879, b'ActiveEnterTimestamp': 1529038632232879, b'ActiveEnterTimestampMonotonic': 2466232879, b'ActiveExitTimestamp': 1529038692242636, b'ActiveExitTimestampMonotonic': 2526242636, b'InactiveEnterTimestamp': 1529038692242636, b'InactiveEnterTimestampMonotonic': 2526242636, b'Job': (0, b'/'), b'ConditionResult': True, b'AssertResult': True, b'ConditionTimestamp': 1529038632228522, b'ConditionTimestampMonotonic': 2466228523, b'AssertTimestamp': 1529038632228524, b'AssertTimestampMonotonic': 2466228524}, []]

the code "looks" complex, but its actually quite easy to follow... i think this accomplish what you need, right?

[aleivag]

i forgot, but i actually "ported" busctl monitor to pystemd in this example https://github.com/facebookincubator/pystemd/blob/master/examples/monitor.py its rudimentary and only use the DBus class, you should probably stick with the example i just gave you in the previous comment, but for educational purposes i mention this here.

[awarecan]

@aleivag Thank you very much for the sample code. I am just following your code and hit that error. Do I have to be root? I don't need that for signal listen in pydbus lib.

pystemd.dbusexc.DBusAccessDeniedError: [err -13]: b'Rejected send message, 1 matched rules; type="method_call", sender=":1.576" (uid=1000 pid=25748 comm="/home/jason/ha/home-assistant/venv/bin/python /hom" label="unconfined") interface="org.freedesktop.DBus.Monitoring" member="BecomeMonitor" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus)'

[awarecan]

Googling told me that: ref

Unfortunately, the default D-Bus policy (at least on Ubuntu) prevents most of the messages (except signals) that goes through system bus from being viewable by dbus-monitor

To change that we need to set a global policy to be able to eavesdrop anything after the individual /etc/dbus-1/system.d/*.conf files applied their restrictions,

[aleivag]

@awarecan yes you have to be root, i dont think you can subscribe to the system event bus with a regular user (but i may be wrong). if your service belong to the user bus (e.g started with systemd-run --user) you can pass True to DBus class constructor to ask for a user bus.

pystemd rely heavily in systemd/sd_bus.h so all integration with dbus are done by systemd internal libraries.

When run as root i can see mayor events, maybe you can share how you do it with pybuslib and i check if anything like that is possible with pystemd.

best regards!

[awarecan]

@aleivag here is my implementation base on pydbus, it doesn't need root, and passed on Ubuntu.

Subscribe to signal https://github.com/awarecan/home-assistant/blob/a9dd02ff0c0a1c9091f3908030e2ad0d28764ae4/homeassistant/components/binary_sensor/systemd.py#L172-L191

Run GLib MainLoop https://github.com/awarecan/home-assistant/blob/a9dd02ff0c0a1c9091f3908030e2ad0d28764ae4/homeassistant/components/binary_sensor/systemd.py#L118-L137

[aleivag]

cool, thanks @awarecan i will look at it in the afternoon and see if i can came up with ideas

thank you for sharing this!!

[aleivag]

hey, so the good news is that it can be done :tada: ... the bad-ish news is that i need to expose sd_bus_match_signal (https://github.com/systemd/systemd/blob/master/src/systemd/sd-bus.h#L362 ) from systemd/sd-bus to pystemd...

i was planing to do it either way, the hard part about that is that its implemented as callbacks, and python callbacks in c land are not as trivial as they sound basically because c has no concept of exception, so a python exception can became a c memory leak.

once i finish coding this, the result code may look like

import select
from pystemd.dbuslib import DBus
from pystemd.systemd1 import Unit

class STATE:
    EXIT = False

def process(msg, err=None):
    print(msg.body)
    if msg.body[1].get(b"SubState") in (b"exited", b"failed", b"dead"):
        STATE.EXIT = True

with DBus(False) as dbus:
    myunit = Unit('mysleep.service')
    dbus.match_signal(
        myunit.destination,
        myunit.path,
        b"org.freedesktop.DBus.Properties",
        b"PropertiesChanged",
        process, #  <-- your function
        None, #  maybe pass custom python objects as userdata?
    )

    fd = dbus.get_fd()
    while not STATE.EXIT:
        select.select([fd], [], []) # wait for message
        dbus.process() # execute the whole message

please notice that that is just an idea, it may end up looking a little bit different. will post here when the code is merged

[aleivag]

hey @awarecan thanks for the patience... the code has been merged, you can test it... a working example can be found in https://github.com/facebookincubator/pystemd/blob/master/examples/monitor_from_signal.py

i test this, and i can see properties changes on units from the system bus without been root...

Hope this helps... (please notice that the example has detail implementation, that may or may not fit your project)... :D

[aleivag]

Heads up! by implementing it as sd_bus_match_signal i made the code only run on systemd v237 and above.

[awarecan]

I am following your example, had met several bumps

1. The callback parameter only accept function, not method.

2. Several different error I got

   Assertion 'm->n_ref > 0' failed at ../src/libsystemd/sd-bus/bus-message.c:934, function sd_bus_message_unref(). Aborting.
   Aborted (core dumped)

corrupted double-linked list
Aborted (core dumped)

3. How to monitor several Unit at same time? Do I need several loop?

[awarecan]

For Assertion 'm->n_ref > 0' failed issue, I already reduced my code to very simple level, but it still happens when I stop or start service. Meanwhile the try/catch didn't work, the whole program crashed

        def process_message(message, error=None, userdata=None):
            pass

        # some other codes

            try:
                with DBus() as bus:
                    bus.match_signal(
                        b'org.freedesktop.systemd1',
                        b'/org/freedesktop/systemd1/unit/bluetooth_2eservice',
                        b'org.freedesktop.DBus.Properties',
                        b'PropertiesChanged',
                        process_message,
                    )
                    fd = bus.get_fd()
                    while True:
                        select.select([fd], [], [])
                        bus.process()
            except Exception as error:
                _LOGGER.error(error)

My systemd is 237 by the way.

> systemd --version
systemd 237
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid

[aleivag]

yeah segfaults are not catchable... that is super weird, let me try to reproduce...

[aleivag]

so i was not able to reproduce, but i'm running v238, will try get a hold of a 237 machine to test, so far the only theory i have is that the dealloc methoc is breaking you. https://github.com/facebookincubator/pystemd/blob/master/pystemd/dbuslib.pyx#L77-L78

you can even try to remove it yourself, just keep in mind that that is the only thing that is preventing a big memory leak on each new message

anyway i will do more digging one a have a v237 in my hands!

[awarecan]

Shall we follow the principle "whoever alloc the memory should dealloc it" (Don't know if there is a fancy word for this) here?

https://github.com/facebookincubator/pystemd/blob/ef980e702053555a3f1577be612c34a54630fda2/pystemd/dbuslib.pyx#L776-L782

m was alloc-ed in libsystemd, so that we should not dealloc it IMO.

[awarecan]

Comment out

https://github.com/facebookincubator/pystemd/blob/ef980e702053555a3f1577be612c34a54630fda2/pystemd/dbuslib.pyx#L77-L78

Everything worked

[aleivag]

i'm both happy (that commenting out makes it work) and sad (i have to make this work on systemd <237 and make a clever de-allocation).

The reason why we did the deallocate, is because normally "we allocated" the msg struct. in the callback case that is not the case anymore, since we have no real way of allocating the struct and tehn hand it over to systemd... i have some ideas, but will sleep on them...

thanks for helping debug!

[aleivag]

hi @awarecan :+1: so i was finally able to reliable reproduce your issue on systemd v237. the issue happens because you dont call msg.process_reply(True). so this happen when your callback is

def process_message(message, error=None, userdata=None):
            pass

and not when your callback is

def process_message(message, error=None, userdata=None):
            message.process_reply(True)

when i implemented this, my logic was to give the user the chance to call msg.process_reply(True) or msg.process_reply(False). the diference is that True will also process the headers. i think that i was wrong and i should give the callback always a processed reply with headers . i'll put a change to do that... will probably do it tomorrow morning tho .

with that change you will not see the problem, and the we return the ownership of the msg to the signal_callback .

note: to reproduce i got a debian strech and isntalled systemd with the backports

Thanks!

[awarecan]

It sounds right. I don't have access to my work now, but I called msg.process_reply(False) actually.