Apprise integration fails to set up multiple services

jimbob1001 commented 8 months ago

The problem

Upon updating to 2024.2 the Apprise notification service fails to set up multiple services. I normally have 12 services created at startup but it now only creates 1 service, seemingly at random (all pushover URLs) (Ironically this was the very problem that used to occur with the native pushover integration which caused me to start using apprise instead)

This worked fine with the previous version (I guess it was 2023.12.something?) and the issue was immediately noticeable upon updating as I received my shutdown notification but did not receive my startup notification

I noticed the docs say there is only 1 active apprise user 😂 maybe I need to go back to the native pushover integration (assuming that now allows multiple services?)

Let me know if I missed anything useful

Cheers James

What version of Home Assistant Core has the issue?

core-2024.2.1

What was the last working version of Home Assistant Core?

core-2023.12.?

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Apprise

Link to integration documentation on our website

https://www.home-assistant.io/integrations/apprise

Diagnostics information

No response

Example YAML snippet

notify:
- name: 'pushover_apprise_1'
  platform: apprise
  url: pover://abcdefg@stuvwxyz

- name: 'pushover_apprise_2'
  platform: apprise
  url: pover://bcdefga@ztuvwxys

Anything in the logs that might be useful for us?

Logger: apprise
Source: components/apprise/notify.py:55
First occurred: 07:19:06 (11 occurrences)
Last logged: 07:19:06

Unparseable URL pover://g...i@a...v
Unparseable URL pover://g...i@a...a
Unparseable URL pover://g...i@a...w
Unparseable URL pover://g...i@a...t
Unparseable URL pover://g...i@a...f

----------

Logger: homeassistant.components.apprise.notify
Source: components/apprise/notify.py:56
Integration: apprise (documentation, issues)
First occurred: 07:19:06 (11 occurrences)
Last logged: 07:19:06

Invalid Apprise URL(s) supplied

----------

Logger: homeassistant.components.notify
Source: components/notify/legacy.py:104
Integration: Notifications (documentation, issues)
First occurred: 07:19:07 (11 occurrences)
Last logged: 07:19:07

Failed to initialize notification service apprise

Additional information

No response

home-assistant[bot] commented 8 months ago

Hey there @caronc, mind taking a look at this issue as it has been labeled with an integration (apprise) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `apprise` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign apprise` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

apprise documentation apprise source _{^{(message by IssueLinks)}}

caronc commented 8 months ago

I'll see if I can investigate, but there have been no changes to Apprise related to Pushover or how Apprise itself loads notifications for a few years now. I suspect it's something new that got introduced into your environment when you upgraded.

Can you test Apprise from the command line for me?

pip install apprise

apprise -vvv -b test "pover://url1" "pover://url2"

jimbob1001 commented 8 months ago

So I did that and successfully received both notifications, however homeassistant still only loads 1 out of 12 services

caronc commented 8 months ago

Odd, so it's something with the environment specifically in HA.

2 things i can think of off hand:

check out this blog i wrote a while ago which configures your Apprise configuration in one spot and can assign it tags. This route loads your configuration in 1 go (and in the same instance). This may or may not help
can you try reapplying the HA update? Something might have gone wrong with it and left over .pyc files lingered?

The other thing could be a change made to how HA uses its plugins from this build and the last was introduced and you're the first to report an issue... I'm not sure.

burgeda commented 8 months ago

I can confirm that Apprise is broken with HA 2024.2.1 I have exactly the same behaviour. I rolled back two weeks, everything worked as expected ... right up until applying 2024.2.1 Version 2024.1.5 works fine.

caronc commented 8 months ago

@frenck : any changes to how asynchronous works or the way Apprise was adapted? With the current version working completely fine on it's own, I'm at a loss what might have been introduced in this latest HA release. I would love your advice! 🙏

LordShadowen commented 8 months ago

I can confirm the same problem.

I have 3 apprise configurations, one loads, the other two fail. Everything was working fine before upgrading.

I've found this in the logs:

2024-02-12 01:50:09.230 WARNING (SyncWorker_4) [apprise] Unsupported schema http.
2024-02-12 01:50:09.230 ERROR (SyncWorker_4) [homeassistant.components.apprise.notify] Invalid Apprise config url provided
2024-02-12 01:50:09.230 WARNING (SyncWorker_4) [apprise] Unsupported schema http.
2024-02-12 01:50:09.230 ERROR (SyncWorker_4) [homeassistant.components.apprise.notify] Invalid Apprise config url provided
2024-02-12 01:50:09.255 ERROR (MainThread) [homeassistant.components.notify] Failed to initialize notification service apprise
2024-02-12 01:50:09.255 ERROR (MainThread) [homeassistant.components.notify] Failed to initialize notification service apprise

Based on a hunch, I swapped my configurations around in the config, and I can confirm that it's always the first one that loads, regardless. There is nothing special about the actual configs, urls, etc... it's something that breaks while going through the YAML after the first one is processed.

daniel-stockhausen commented 8 months ago

Same issue, 5 apprise services - 1 loads 4 fail.

2024-02-18 06:32:29.717 INFO (MainThread) [homeassistant.components.notify] Setting up notify.apprise
2024-02-18 06:32:29.717 INFO (MainThread) [homeassistant.components.notify] Setting up notify.apprise
2024-02-18 06:32:29.718 INFO (MainThread) [homeassistant.components.notify] Setting up notify.apprise
2024-02-18 06:32:29.730 ERROR (SyncWorker_4) [apprise] Unparseable URL pover://L...i@a...m?sound=intermission
2024-02-18 06:32:29.730 ERROR (SyncWorker_4) [homeassistant.components.apprise.notify] Invalid Apprise URL(s) supplied
2024-02-18 06:32:29.731 ERROR (SyncWorker_2) [apprise] Unparseable URL pover://L...i@a...m?sound=bike
2024-02-18 06:32:29.731 ERROR (SyncWorker_2) [homeassistant.components.apprise.notify] Invalid Apprise URL(s) supplied
2024-02-18 06:32:29.732 INFO (MainThread) [homeassistant.components.notify] Setting up notify.apprise
2024-02-18 06:32:29.732 INFO (MainThread) [homeassistant.components.notify] Setting up notify.apprise
2024-02-18 06:32:29.733 ERROR (SyncWorker_6) [apprise] Unparseable URL pover://L...i@a...z?priority=emergency&expire=10800&sound=bike
2024-02-18 06:32:29.733 ERROR (SyncWorker_6) [homeassistant.components.apprise.notify] Invalid Apprise URL(s) supplied
2024-02-18 06:32:29.734 ERROR (SyncWorker_6) [apprise] Unparseable URL tgram://somenumber:someid/somenumber/
2024-02-18 06:32:29.734 ERROR (SyncWorker_6) [homeassistant.components.apprise.notify] Invalid Apprise URL(s) supplied

caronc commented 8 months ago

Analysis

So as it turns out HA changed around how it handles it's configuration files. I was able to reproduce this:

When i provided:

# Defining 2 URLs
notify:
  - name: test1
    platform: apprise
    url: ntfys://credentials@my.host/queue1, pover://pocredentials

HA now no longer treats the url element as a string as it did in previous versions. Hence Apprise expected:

ntfys://credentials@my.host/queue1, pover://pocredentials :point_right: type(str)

But what it got was a list containing 1 element in it as:

['ntfys://credentials@my.host/queue1, pover://pocredentials'] :point_right: type(list)

Apprise does recursively look into strings from within of defined arrays/lists, the unfortunate expected result is now:

2024-02-18 15:06:26.553 WARNING (SyncWorker_2) [apprise] A specified ntfy topic (pover:) is invalid and will be ignored
2024-02-18 15:06:26.553 WARNING (SyncWorker_2) [apprise] A specified ntfy topic (pocredentials) is invalid and will be ignored

Workaround

With the new changes to HA: the workaround (for now) would be to change your Apprise URL to this:

  - name: test1
    platform: apprise
    url:
     - ntfys://credentials@my.host/queue1
     - pover://pocredentials

The above returns the a with 2 URL elements in it; Apprise can very easily parse this.

Secret Conundrum

Unfortunately this does not solve the problem where people may have configured the following in their config file:

  - name: test1
    platform: apprise
    url: !my_secret

Those with secrets for now will have to brake their URL secrets into multiple entries unfortunately and do the following using the new HA release:

  - name: test1
    platform: apprise
    url:
      -  !my_single_url_secret1
      -  !my_single_url_secret2
      -  !my_single_url_secret3

Logic Changes Under Hood

It seems its related to this line vol.Optional(CONF_URL): vol.All(cv.ensure_list, [str]),. It looks like it's been enforcing a list for a long time but not receiving one until this latest release when it all broke.

Fix is on it's way

I have a PR coming that will restore the way Apprise worked in the past using the new lists it receives. Once the PR is in place, your old logic in place will work fine. Until then, you'll need to use the workaround defined here.

gilcu3 commented 8 months ago

@caronc Just to confirm, does the fix proposed solves the initial issue, where several apprise services are defined and only the first works (they lead to different notify.servicename), or just the issue when a single service uses several urls? These uses cases seems different to me at least.

caronc commented 8 months ago

I'll need to investigate, but there is a good chance it did🚀

caronc commented 8 months ago

My colleague at work mentioned this didn't work for some configuration; specifically the one involving the spread configuration files (also something that worked fine 3 releases ago). Can anyone confirm?

I'm traveling for work, but i can try to investigate further when i return next week)

daniel-stockhausen commented 8 months ago

I still got the issue with 2024.2.3 and this configuration:

notify:
  - platform: apprise
    name: pushover
    url: !secret apprise_pushover_default
  - platform: apprise
    name: pushover_critical
    url: !secret apprise_pushover_critical
  - platform: apprise
    name: telegram_log_error
    url: !secret apprise_telegram_log_error

gilcu3 commented 8 months ago

My colleague at work mentioned this didn't work for some configuration; specifically the one involving the spread configuration files (also something that worked fine 3 releases ago). Can anyone confirm?

I'm traveling for work, but i can try to investigate further when i return next week)

My configuration simply has two apprise entries, each with a single url and different name, and it was working until 2 weeks ago and does not work in current version (only the first entry works).

Tuncay-Ayhan commented 8 months ago

I still got the issue with 2024.2.3 and this configuration:

notify:
  - platform: apprise
    name: pushover
    url: !secret apprise_pushover_default
  - platform: apprise
    name: pushover_critical
    url: !secret apprise_pushover_critical
  - platform: apprise
    name: telegram_log_error
    url: !secret apprise_telegram_log_error

Same for me. Even on 2024.02.4

caronc commented 8 months ago

I'm sorry all 😔. I'll investigate when i get home next week.

At this point I'll need to start tracking down what else changed with Home Assistant in its recent releases. Hopefully it will be straight forward to adapt. 🙏

caronc commented 8 months ago

See PR attached to explain what was happening to you all. I added unit tests in Apprise as well to catch this kind of thing for the future as well.

The short of it is: It turns out to be a racing condition that was introduced in Apprise v1.7.0 where (notification) modules were only loaded on demand. As it turns out, I didn't consider or test the case where there would be multiple asynchronous calls to it's initialization. Hence for each notify Apprise instance you guys had declared tried to initialize Apprise uniquely in another thread.

The new (Apprise) release I'll prepare is v1.7.3 which will include the above PR and fix this. Once that is done, a small PR to Home Assistant will be needed to bump the version; no more complicated then the past ones made (e.g. 1.7.2 , 1.6.0, etc)

for those who can't wait: the (Apprise) master branch works great. If you're programmer-savvy, you just need to update your HA environment to grab it:

# Grab latest (patched) Apprise from Source
pip3 install git+https://github.com/caronc/apprise.git --upgrade --force

Edit: Created Apprise v1.7.3 Release which should resolve the issue we're seeing here once HA is updated to reference it 🤞

home-assistant / core