Graphical Notifications Stop Appearing On-Screen, but --command Flag Still Works (Custom systemctl Unit file)

[Ao1Pointblank]

Description

Notifications sometimes do not send, but the --command will run just fine. This may be related to the service dying, but only partially? I don't understand it.

Reproduction

I use Linux Mint Cinnamon, all up-to-date.

The command i use (added in Cinnamon's Startup Applications): /home/pointblank/.local/bin/a2ln --pairing-port 46351 --notification-port 46352 --title-format "{app} | {title}" --command 'if [ -f ~/.sounds/a2ln/{app}.mp3 ]; then play -Gq --norm=0 ~/.sounds/a2ln/{app}.mp3; else play -Gq --norm=0 ~/.sounds/a2ln/default.mp3; fi'

an additional system file: Located at: /home/pointblank/.config/systemd/user/a2ln.service Started by: systemctl --user enable --now a2ln (should only need to be done once)

[Unit]
Description=Android 2 Linux Notifications Server

[Service]
ExecStart=/home/pointblank/.local/bin/a2ln --pairing-port 46351 --notification-port 46352 --title-format "{app} | {title}" --command 'if [ -f ~/.sounds/a2ln/{app}.mp3 ]; then play -Gq --norm=0 ~/.sounds/a2ln/{app}.mp3; else play -Gq --norm=0 ~/.sounds/a2ln/default.mp3; fi'
RuntimeMaxSec=20m
Restart=always

[Install]
WantedBy=graphical.target

It is worth noting that this unit file attempts to restart the server every 20 minutes. This is my workaround to the server dying periodically and without warning, which resulted in unreliable notifications (unacceptable)

When I notice that the service was partially broken and not showing notifications on screen, and only running the --command (which plays a sound effect), I normally restart it with systemctl --user restart a2ln.service However, this time I remembered to check the status first: $ systemctl --user status a2ln.service :

a2ln.service - Android 2 Linux Notifications Server
     Loaded: loaded (/home/pointblank/.config/systemd/user/a2ln.service; enabled; vendor preset: enabled)
     Active: failed (Result: start-limit-hit) since Sat 2023-05-27 22:43:47 EDT; 1 day 15h ago
    Process: 3272658 ExecStart=/home/pointblank/.local/bin/a2ln --pairing-port 46351 --notification-port 46352 --title-format {app} | {title} --command if [ ->
   Main PID: 3272658 (code=exited, status=0/SUCCESS)
        CPU: 99ms

May 27 22:43:47 monolith systemd[1702]: a2ln.service: Scheduled restart job, restart counter is at 2490.
May 27 22:43:47 monolith systemd[1702]: Stopped Android 2 Linux Notifications Server.
May 27 22:43:47 monolith systemd[1702]: a2ln.service: Start request repeated too quickly.
May 27 22:43:47 monolith systemd[1702]: a2ln.service: Failed with result 'start-limit-hit'.
May 27 22:43:47 monolith systemd[1702]: Failed to start Android 2 Linux Notifications Server.

The message "Start request repeated too quickly." suggests that it is my unit file's 20min restart timer causing an error. But if i change that, then I won't even get sounds when it dies randomly. What should I do? (I will attempt reinstalling A2LN and using the default Unit file / recommended install method until I can confirm again that it does not work reliably. Maybe I am forgetting something...)

[Ao1Pointblank]

I edited my custom Unit file to add the parameter "RestartSec=5s" (followed by systemctl --user daemon-reload) This should prevent it from starting up too fast after a failure. I also killed the running instance with pkill, and then stopped the service with systemctl --user stop a2ln.service (I noticed I have to forcefully kill the server when stopping it, or it will hog the port and prevent a new instance from starting)

So, after starting it with systemctl --user start a2ln.service, and checking systemctl --user status a2ln.service to ensure it is reporting as 'active', I opened the android app and sent a notification: no dice. the custom sound effect plays, but no on-screen message.

Other notifications like Discord, Mailspring, and plain old notify-send commands work fine.

[patri9ck]

It is worth noting that this unit file attempts to restart the server every 20 minutes. This is my workaround to the server dying periodically and without warning, which resulted in unreliable notifications (unacceptable)

So the A2LN server just randomly dies after some time and you have to restart it?

[Ao1Pointblank]

It is worth noting that this unit file attempts to restart the server every 20 minutes. This is my workaround to the server dying periodically and without warning, which resulted in unreliable notifications (unacceptable)

So the A2LN server just randomly dies after some time and you have to restart it?

yes, exactly. well, kindof. if the server was completely dead, the signal to run --command wouldn't be transmitted. so maybe it is something else related to the Unit file. I've done some more testing, and came up with this:

[Unit]
Description=Android 2 Linux Notifications Server

[Service]
ExecStart=/home/pointblank/.local/bin/a2ln --pairing-port 46351 --notification-port 46352 --title-format "{app} | {title}" --command 'if [ -f ~/.sounds/a2ln/{app}.mp3 ]; then play -Gq --norm=0 ~/.so>
RuntimeMaxSec=1h
Restart=on-failure
RestartSec=20s

[Install]
WantedBy=default.target

It seems to be working, restarting automatically every hour, with a 20s pause in between attempts. My guess is that when a random crash occurs, it will be unable to reopen to server due to the port being hogged by the broken process. I also don't know if it will work properly after rebooting. We'll see. Any ideas until I find out more?

[patri9ck]

yes, exactly. well, kindof. if the server was completely dead, the signal to run --command wouldn't be transmitted.

So that is why you use a custom systemd unit file - to restart the server periodically because no notification are shown after some time.

I guess instead of working on the systemd unit file then, we should rather work on the actual problem that no notifications are being shown. Right now though, I have no idea why this might be happening.

[Ao1Pointblank]

yes, exactly. well, kindof. if the server was completely dead, the signal to run --command wouldn't be transmitted.

So that is why you use a custom systemd unit file - to restart the server periodically because no notification are shown after some time.

I guess instead of working on the systemd unit file then, we should rather work on the actual problem that no notifications are being shown. Right now though, I have no idea why this might be happening.

I'm sorry I can't provide much helpful data right now. I will clear my logs and report back if it happens again. will output from journalctl --user -u a2ln.service suffice?

[patri9ck]

Does this issue still persist?

[Ao1Pointblank]

Does this issue still persist?

yes, sometimes i still hear the custom sound effect specified in the --command flag, but no graphical notify-send popup until i killall a2ln and relaunch it.

[Unit]
Description=Android 2 Linux Notifications Server

[Service]
#ExecStart=/home/pointblank/.local/bin/a2ln
ExecStart=/home/pointblank/.local/bin/a2ln "--port" "23045" "--title-format" "{app} | {title}" "--command" "if [ -f ~/.sounds/a2ln/\"{app}\".mp3 ]; then play -Gq --norm=-4 ~/.sounds/a2ln/\"{app}\".mp3; else play -Gq --norm=-4 ~/.sounds/a2ln/default.mp3; fi"
[Install]
WantedBy=default.target

[iinuwa]

My guess is that when a random crash occurs, it will be unable to reopen to server due to the port being hogged by the broken process.

This sounds like when sockets are opened without using the SO_REUSEADDR option. A quick search shows that there is a similar option for ZeroMQ: ZMQ_LINGER. You could experiment with that.

[Ao1Pointblank]

My guess is that when a random crash occurs, it will be unable to reopen to server due to the port being hogged by the broken process.

This sounds like when sockets are opened without using the SO_REUSEADDR option. A quick search shows that there is a similar option for ZeroMQ: ZMQ_LINGER. You could experiment with that.

i have made a modified version which implements --linger as an option as you suggested. is it required for both the pairing port and the notification port or just the latter?

i have it with both right now, with a value of 5000ms and it seems to be working but i will have to test / let the server idle for a few days just in case.

[Ao1Pointblank]

assuming i implemented ZMQ_LIGER correctly, this is not the solution. however, I did get some errors:

● a2ln.service - Android 2 Linux Notifications Server
     Loaded: loaded (/home/pointblank/.config/systemd/user/a2ln.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2024-06-23 22:19:02 EDT; 3 days ago
   Main PID: 185080 (a2ln)
      Tasks: 8 (limit: 18898)
     Memory: 13.7M
        CPU: 6.796s
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/a2ln.service
             └─185080 /usr/bin/python3 /home/pointblank/.local/bin/a2ln --linger 5000 --port 23045 --title-format "{app} | {title}" --command "if [ -f ~/.sounds/a2ln/\"{app}\".mp3 ]; then play -Gq --norm=-4 ~/.sounds/a2ln/\"{app}\".mp3; else play -Gq --norm=-4 ~/.sounds/a2ln/default.mp3; fi"

Jun 27 15:30:08 monolith a2ln[185080]: gi.repository.GLib.GError: g-dbus-error-quark: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name :1.45 was not provided by any .service files (2)
Jun 27 15:30:19 monolith a2ln[185080]: Exception in thread Thread-52 (send_notification):
Jun 27 15:30:19 monolith a2ln[185080]: Traceback (most recent call last):
Jun 27 15:30:19 monolith a2ln[185080]:   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Jun 27 15:30:19 monolith a2ln[185080]:     self.run()
Jun 27 15:30:19 monolith a2ln[185080]:   File "/usr/lib/python3.10/threading.py", line 953, in run
Jun 27 15:30:19 monolith a2ln[185080]:     self._target(*self._args, **self._kwargs)
Jun 27 15:30:19 monolith a2ln[185080]:   File "/home/pointblank/.local/lib/python3.10/site-packages/a2ln/a2ln.py", line 131, in send_notification
Jun 27 15:30:19 monolith a2ln[185080]:     Notify.Notification.new(title, body, "dialog-information").show()
Jun 27 15:30:19 monolith a2ln[185080]: gi.repository.GLib.GError: g-dbus-error-quark: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name :1.45 was not provided by any .service files (2)

the ExecStart value in the systemd file:

/home/pointblank/.local/bin/a2ln \
--linger 5000 --port 23045 \
--title-format "{app} | {title}" \
--command "if [ -f ~/.sounds/a2ln/\"{app}\".mp3 ]; then play -Gq --norm=-4 ~/.sounds/a2ln/\"{app}\".mp3; else play -Gq --norm=-4 ~/.sounds/a2ln/default.mp3; fi"

and here is a modified version of the script with the --linger option: https://gist.github.com/Ao1Pointblank/3330c2a7a1cb85cbd864d2aec0d47620

[patri9ck]

Does this issue still persist? Does it also happen when the server is auto-started by one of the other options instead of systemd?