search

openhab / openhab-addons

Add-ons for openHAB
https://www.openhab.org/
Eclipse Public License 2.0
1.86k stars 3.56k forks source link

[openhabcloud] Support new notification features #16934

Open digitaldan opened 5 days ago

digitaldan commented 5 days ago

Mobile Notification Enhancements

Using this ticket to coordinate changes between our various projects to support new notifications features. This includes:

Message Payload Keys (Proposal)

Action Format

An action can be either a command, ui, http or https action.

Command Action

command:itemName:commandString

Example: command:KitchenLights:ON

UI Action

ui:$itemCommandSyntax

where $itemCommandSyntax is the same syntax as defined at https://next.openhab.org/docs/mainui/about.html#ui-command-item

Example

OR

ui:$path

where $path is either /basicui/app?..... for sitemaps (native sitemap renderer will be used) or /some/absolute/path for navigating the view. $path in either case starts with a /

Example

http | https action

https://openhab.org

This will open the link in a embedded browser, and is meant for loading external web pages.

Sample Payload

This is what may be sent from the binding to openHAB Cloud

{
  "media-attachment-url": "https://upload.wikimedia.org/wikipedia/commons/thumb/0/07/OpenHAB_logo_2.svg/541px-OpenHAB_logo_2.svg.png",
  "message": "Test Notification",
  "title": "openHAB",
  "on-click": "ui:popup:widget:myKitchenWidget",
  "actions": [
    {
      "title": "Turn Kitchen Lights ON",
      "action": "command:KitchenLightst:ON"
    },
    {
      "title": "Turn Kitchen Lights OFF",
      "action": "command:KitchenLights:OFF"
    },
    {
      "title": "View Kitchen",
      "action": "ui:navigate:/page/Kitchen"
    }
  ]
}

Associated Tickets

Android: https://github.com/openhab/openhab-android/issues/3193 (original discussion here) IOS: https://github.com/openhab/openhab-ios/pull/764 Cloud: https://github.com/openhab/openhab-cloud/pull/462 MainUI: https://github.com/openhab/openhab-webui/pull/2622 Cloud Addon: #16938 openhab-js: https://github.com/openhab/openhab-js/pull/351 Docs: #16941 and https://github.com/openhab/openhab-docs/pull/2320

TODO

Checklist for implementation:

Notify maintainers of other UIs: @openhab/webui-maintainers @openhab/android-maintainers @openhab/ios-maintainers

Sample Screen Shot (IOS) IMG_5C6672F754A4-1

maniac103 commented 4 days ago

Does iOS limit the number of actions? Buttons in Android are limited to a maximum of 3, FWIW, so if iOS has a similar limitation, we may want to state that in the documentation of the feature. Also the action text must be way shorter on Android (one word essentially), but I guess the users will figure that one out.

digitaldan commented 4 days ago

Its a good question, a quick search shows that its also 3 for IOS , not sure what happens when you try adding more then that, guess i'll test, but yes thats a good point, and will need to document.

~FYI, i have a central issue to centralize our discussions, https://github.com/openhab/openhab-addons/issues/16934~ Edited : Sorry thought this comment was on a different thread !

florian-h05 commented 4 days ago

If we get the required change to the openHAB Cloud Connector add-on done before the feature freeze, it would be possible to roll out the new notifications after the 4.2.0 release as we then only need to change cloud and the apps.

digitaldan commented 4 days ago

I had a similar thought, I think its very possible, I have the work half done, its actually not all that much to change, just adding a new overloaded method with the new fields. I can at least try to get something ready for review today or tomorrow and see how close we are. The cloud code is done, and i plan on pushing that out soon.

seime commented 4 days ago

What I would love to see is the possibility to cancel or retract notifications!

Use case(s): Retract an open door notification when it is closed. Or whenever something that passed a threshold value is reduced below the threshold again like power usage, temperature, humidity etc.

digitaldan commented 4 days ago

In rules, our send notification looks like this:

public static void sendNotification(String userId, String message, @Nullable String icon,
            @Nullable String severity)

I was thinking to extend it to this, especially now that i know there is a max of 3 action buttons you can add on IOS/Android, notice that many parameters are optional (null) 

public static void sendNotification(String userId, String message, @Nullable String icon, @Nullable String severity,
            @Nullable String onClickAction, @Nullable String mediaAttachmentURl, @Nullable String actionButton1,
            @Nullable String actionButton2, @Nullable String actionButton3) {

an action button String would have the following format ButtonTitle=Action, for example Turn Kitchen Lights ON=command:KitchenLightst:ON

Alternatively, we could have one actionButton string which , delimits actions, so ButtonTitle=Action,ButtonTitle=Action, but i kinda like having the 3 since it makes it clearer as @maniac103 pointed out.

florian-h05 commented 4 days ago

I can at least try to get something ready for review today or tomorrow and see how close we are.

Great!

What I would love to see is the possibility to cancel or retract notifications!

Yeah, that would be absolutely amazing, but I think if we are able to ship the actions and images to openHAB 4.2.0 we already have a big improvement, but this would be a nice improvement for openHAB 4.3.0.

In rules, our send notification looks like this:

I like that proposal, makes IMO more sense than the single, big string.

The JS and Ruby helper libraries could create a builder around this method to allow easier usage of it.

digitaldan commented 4 days ago

I like that proposal, makes IMO more sense than the single, big string.

Excellent, I'll go with that then and work on this tonight. Thanks for the feedback !

digitaldan commented 4 days ago

So i have a version of the cloud binding working and ready for a review. Just FYI, I'll probably give it a once over tomorrow, but its working nicely on my system so far. See #16938

florian-h05 commented 4 days ago

BTW I have already created a WIP PR for a notification builder for the openHAB JavaScript library: https://github.com/openhab/openhab-js/pull/351.

digitaldan commented 4 days ago

@mueller-ma @maniac103 quick question, right now notifications have a single message. In IOS notifications have both a Title and Body , we set the Title to openHAB and Body to the value of the message. Should we allow the user to optionally set the Title as well (but would default to openHAB if not set) . Hoping for a quick answer here, would be a small thing to add, but i want to do this (or not) before we merge anything.

Edit, actually IOS we don't set the title, so it defaults to the App name, which is fine if its not set. I updated the cloud code to support this, and the IOS app should respect it, regardless if we add it to the binding or not.

digitaldan commented 4 days ago

Actually i think we should support Title, lets make Notifications as powerful as possible, and i don't want to have to come back and create another overloaded method just to support this in the future.

mueller-ma commented 4 days ago

It's similar to Android: Right now we don't set a title, but only a summary (probably what body is on iOS). Adding an optional title should be fine 👍

digitaldan commented 4 days ago

Great, thanks for the quick feedback. I just updated the binding PR to support this and was able to test end to end with a IOS client.

mueller-ma commented 3 days ago

What about allowing an Image or Video Item name as value for media-attachment-url?

digitaldan commented 3 days ago

What about allowing an Image or Video Item name as value for media-attachment-url?

I was also thinking to allow relative paths like /static/html/.... and we would try hitting the user's openHAB instance. The IOS client code right now is very simple and is just making a dumb http request, i need to implement using the remote/local connection info so we can use myopenhab if needed for proxing this connection .

florian-h05 commented 3 days ago

But this should be client code, not cloud connector, so we should be able to merge the openHAB server changes now.

digitaldan commented 3 days ago

@florian-h05 correct, binding and cloud code should not care. I think we can detect url patterns , maybe even item://imageItem or something like that for advanced use of pulling of images from items from clients.

digitaldan commented 3 days ago

btw, cloud code is deployed, so the binding should work in production

mueller-ma commented 3 days ago

maybe even item://imageItem or something like that for advanced use of pulling of images from items from clients

The item:// prefix should be required when the client handles it like this:

if value startsWith "/" -> relative url
else if value startsWith "http(s);//" -> absolute url
else -> item
digitaldan commented 2 days ago

So I started to actually use this feature on my home system, very very cool

image

ghys commented 1 day ago

@digitaldan it's a great development, I've been wanting to get rich notifications for a long time.

I don't know if you're aware or remember, but I made an implementation of the Web Push API for HABot (see here). Depending on the browser and OS it can also display an image and action buttons in some cases:

https://developer.mozilla.org/en-US/docs/Web/API/Notification/image https://developer.mozilla.org/en-US/docs/Web/API/Notification/actions

My main problem at the time was the authentication - since the web push notification feature didn't need openHAB Cloud at all it was difficult to get more than a basic notification because it would require authenticated access to your instance to get the image or perform the actions.

I wonder if it's the right time to resuscitate this web push thing, only in the main UI, with possibly rich notifications? On mobile you would be better served with the apps, of course, but this could introduce notifications for your desktop or laptop computer as well.

Edit: this Medium article shows how web push notifications with an image and/or actions would look on different environments: https://leemunroe.medium.com/the-state-of-push-notification-design-in-2021-21e2ab7aab5e

digitaldan commented 1 day ago

@ghys i'll definitely read up on browser notifications, its an area i know very little about, but it does sound like an awesome feature, especially if there might be a way to tie it in or unify it with mobile push.

openhab-bot commented 1 day ago

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/jruby-scripting-official-helper-library/145072/15

jimtng commented 1 day ago

Suggestion: for action, add two more:

jimtng commented 1 day ago

Ruby support has been merged https://github.com/openhab/openhab-jruby/pull/285

    notify "Someone pressed the doorbell!",
      title: "Doorbell",
      attachment: "http://myserver.local/cameras/frontdoor.jpg",
      buttons: {
        "Show Camera" => "ui:/basicui/app?w=0001&sitemap=cameras",
        "Unlock Door" => "command:FrontDoor_Lock:OFF"
      }
digitaldan commented 1 day ago

@jimtng those sound like good ideas. The way IOS works, for http(s): i think we would need to bring the app to the foreground first (like we do with ui:) , then we could either launch an external web browser or the internal one. And would be similar for app: as well. I think on IOS, apps need to register themselves to be opened via a URL , so like myApp://somePath?foo=bar , on Android i thought apps are launched via their bundle id, like com.openhab.app, but its been awhile since i have looked into it.

Thanks for getting the ruby support in so quick!

digitaldan commented 21 hours ago

when using image items for the media-attachment-url , how about using item:myItemName ? This can still be parsed as a valid URI in both Swift and Kotlin for easy parsing

Kotlin , when using a URI object, would see this as 

Scheme: item
Host: null
Path: null
Query: null
Port: -1
Scheme Specific Part: myItemName

the item name would be located at uri.schemeSpecificPart

digitaldan commented 20 hours ago

So the IOS app now supports using the item:itemName format for media attachments, i'm waiting on testflight to approve our initial build so we can start getting users on it to test. Once that happens I'll update the IOS readme with the new App link and we can coordinate some messaging on the forums to point users to it.

digitaldan commented 14 hours ago

I updated the actions description to include http/https links in actions , this has also been added to the IOS client.

jimtng commented 13 hours ago

Please update https://github.com/openhab/openhab-addons/blob/main/bundles/org.openhab.io.openhabcloud/README.md as that's my source of reference for this :)

digitaldan commented 1 hour ago

I'll update the docs today, i don't think it should affect the rule libraries other then documentation ? This was strictly a mobile client side implementation.

florian-h05 commented 1 hour ago

It is only a documentation update (in case of JS Scripting I just add a link to the new section for the media attachment url in the cloud add-on docs).