search

MaxvandeLaar / homey-webos-plus

Improved LG WebOS integration for Homey
GNU General Public License v3.0
27 stars 11 forks source link

Sending toastmessage with gallery image not displayed on TV #15

Closed zwaardemaker closed 4 years ago

zwaardemaker commented 4 years ago

Sending toastmessage with gallery image not displayed on TV

Tryed different images and filesizes, but non of the images and text are displayed on TV

MaxvandeLaar commented 4 years ago

@zwaardemaker, what were any special mime-types? Such as gif or something else?

zwaardemaker commented 4 years ago

@MaxvandeLaar , did try gif and jpg

MaxvandeLaar commented 4 years ago

@zwaardemaker I know for sure gif won't work. WebOS won't accept that, jpg should work though. I don't know if it is possible to restrict mime-types when using the gallery images.

I will test jpg again within a few days. Currently I am busy rewriting the entire code base so it has to wait until that is finished.

zwaardemaker commented 4 years ago

@MaxvandeLaar I will try again a small sized jpg and let you know later on! Great app so far!

MaxvandeLaar commented 4 years ago

@zwaardemaker, I just tried it with a .jpeg image and it worked flawless for me.

OllemGit commented 4 years ago

Easy way to test by using 'image grabber app' to get any kind and size of image to test with Homey Doesn't work even with very small jpg for me...

image image

MaxvandeLaar commented 4 years ago

Hmm interesting, I will check what happens here.

MaxvandeLaar commented 4 years ago

@OllemGit can you share an image url that is not working for you, because it works for me.

zwaardemaker commented 4 years ago

Doesn't work with jpeg either overhere🤔

MaxvandeLaar commented 4 years ago

@zwaardemaker @OllemGit, I encountered a not working toast only when the resolution was extreem (2000x1000) for example. The same image, much smaller works without any issues for me. Tried jpeg, jpg, png with Image Grabber, Gallery, and plain Base64, Url with other toast card. Please send me an image to test with, because I cannot reproduce this without going to extreem resolutions.

OllemGit commented 4 years ago

This is the image I used in my test sample flow - I just googled a simple smiley image:

https://c2.plzcdn.com/ZillaIMG/800f03d026e08ab0d6c95baebda55ba8.jpg

image

OllemGit commented 4 years ago

Could it be WebOS version related issue? My TV has WebOS v4.0

MaxvandeLaar commented 4 years ago

@OllemGit Can you try it with this link? https://s3.eu-central-1.amazonaws.com/plzcdn-zillaimg/800f03d026e08ab0d6c95baebda55ba8.jpg

I noticed that your link redirects to the actual image, maybe that's the problem. Note that I have 2 toast cards

image

The first card also accepts an image link (no redirect links) can you try that one instead of the gallery one? They both use the same call to send a toast to the tv. If one of the two works, it makes it easier to find the specific part that causes the problem. WebOS 4 might indeed be the problem but it's hard to say without the ability for me to test one.

zwaardemaker commented 4 years ago

@MaxvandeLaar Can you test this one?

chaos

MaxvandeLaar commented 4 years ago

@zwaardemake I test by sending a toast to my tv and by sending a push notification to my phone.

With image grabber referred to the url https://user-images.githubusercontent.com/25539513/77747003-27f63c80-701e-11ea-8ffa-aff50d9be278.jpg. Image did not work on both the TV and phone.

Using the same link with my toast card in the card without gallery (just text and base64/url) the link does work.

Downloading the image and adding it to the gallery app. It works for both the toast card with gallery and homey push notification with gallery to my phone.

zwaardemaker commented 4 years ago

@MaxvandeLaar think it tv related notification from gallery to my phone works fine. From gallery to toast and from the base 63url it doesn't work

OllemGit commented 4 years ago

I quickly tested and including image URL using the regular toast message doesn't work either. With just plain text it does work.

zwaardemaker commented 4 years ago

Also overhere. Just a textmessage works fine

MaxvandeLaar commented 4 years ago

@OllemGit & @zwaardemaker both webos 4.x?

OllemGit commented 4 years ago

My set is at WebOS 4.0 yes

zwaardemaker commented 4 years ago

Not Sure. I have an oled65b8. Standard with webos 3.5 i think. Had some updates. Softwareversion 5.10.40

MaxvandeLaar commented 4 years ago

Okay, so this seems very hard to fix at the moment. What I'll do is, try to make some changes in my code which send the mime-type to the LG Tv. I will let you guys know here when I released the version with the fix. If this does not seem to do anything for you, I don't really know how to solve the issue and I cannot promise you I can/will fix it anytime soon.

Sorry for this somewhat disappointing conclusion.

zwaardemaker commented 4 years ago

It's not the and of the world. Thank you so much for now! Let me know when you made the changes and i will test it

OllemGit commented 4 years ago

+1 Not urgent and thanks for your efforts!

MaxvandeLaar commented 4 years ago

@zwaardemaker @OllemGit I made changes to getting the mime-type in https://github.com/MaxvandeLaar/homey-webos-plus/releases/tag/v2.0.0-rc2. Please note that this is a release candidate and not a production release just yet. Mostly changed the way power on/off is detected and includes a complete rewrite of the code with much more logging to help debugging.

zwaardemaker commented 4 years ago

I will test it asap!

MaxvandeLaar commented 4 years ago

@zwaardemaker @OllemGit Already released a new candidate with on/off listener improvements. Best to just download the latest (pre)release version to test with.

zwaardemaker commented 4 years ago

@MaxvandeLaar first thing tomorrow morning!

zwaardemaker commented 4 years ago

@MaxvandeLaar both methods still don't work for me.

MaxvandeLaar commented 4 years ago

@zwaardemaker @OllemGit, Could you try out the release candidate 6?

https://github.com/MaxvandeLaar/homey-webos-plus/releases/tag/v2.0.0-rc6

It appears the toast message API never throws an error even though there is one... So now I figured out how to catch them I show them when it fails in the flow editor. I also resized all images to 200px to prevent images which are to large.

OllemGit commented 4 years ago

I just did but no luck at all when sending a simple txt toast message the app crashes it seems? Happened twice with just sending a very simple 'text toast message'

✓ Pre-processing app... ✓ Validating app... ✓ Homey App validated successfully against level debug ✓ Packing Homey App... ✓ Pruning dev dependencies... — App size: 8.41 MB ✓ Installing Homey App on Homey (http://192.168.2.117:80)... ✓ Homey App com.maxvandelaar.webos-plus successfully installed ✓ Running com.maxvandelaar.webos-plus, press CTRL+C to quit — Profile your app's performance at https://go.athom.com/app-profiling?homey=56dd75684a861e1678e244a1&app=com.maxvandelaar.webos-plus ─────────────── Logging stdout & stderr ─────────────── ✖ Connection has been lost, exiting...

OllemGit commented 4 years ago

Tried again and this time got error message:

The app did not crash or loose connecttion this time. I could e.g. switch TV off and on using the app. Just the flow failed.

2020-04-02 18:00:47 [log] [ManagerDrivers] [webos_plus] [0] powerStateListener: received on 2020-04-02 18:00:57 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as undefined unhandledRejection TypeError: First argument must be a string, Buffer, ArrayBuffer, Array, or array-like object. at Function.Buffer.from (buffer.js:183:11) at new Buffer (buffer.js:158:17) at Promise (/drivers/webos_plus/device.js:1:142716) at new Promise () at _0x581326.exports._resizeIcon (/drivers/webos_plus/device.js:1:142664) at Promise (/drivers/webos_plus/device.js:1:141825) at new Promise () at _0x581326.exports._toastSend (/drivers/webos_plus/device.js:1:141721) at Promise (/drivers/webos_plus/driver.js:1:36032) at new Promise ()

zwaardemaker commented 4 years ago

No crashes overhere. No error, but also no message and no image either.

MaxvandeLaar commented 4 years ago

@OllemGit ohh I forgot to check for empty icon... sorry will fix that soon...

MaxvandeLaar commented 4 years ago

@OllemGit I fixed the crashing https://github.com/MaxvandeLaar/homey-webos-plus/releases/tag/v2.0.0-rc7

@zwaardemaker @OllemGit could you try again and preferably check the logging as well to see what the response is? Just paste the log in here so I can check it

You can see the logging by running the app instead of installing using homey app run. When you are finished reading the log you can install the app normally homey app install. It would definitely help me understand better what is happening

OllemGit commented 4 years ago

with rc7 same test without image now runs successfully again:

2020-04-02 20:00:29 [log] [ManagerDrivers] [webos_plus] [0] appListener: Set image for 'com.webos.app.hdmi1' HDMI 1 (http://192.168.2.x:3000/resources/e150e19f45339af2f497a051337040e37108ccda/HDMI_1.png) 2020-04-02 20:00:30 [log] [ManagerDrivers] [webos_plus] [0] _powerStateListener: Called timeout active null 2020-04-02 20:00:30 [log] [ManagerDrivers] [webos_plus] [0] powerStateListener: received on 2020-04-02 20:00:53 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as undefined 2020-04-02 20:00:53 [log] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast 2020-04-02 20:00:53 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Successfully sent a toast message { returnValue: true, toastId: 'HUUE7GGx.754-1585850452664' }

OllemGit commented 4 years ago

With image (some internet jpg) I get following error message:

2020-04-02 20:03:15 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as undefined 2020-04-02 20:03:15 [log] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast 2020-04-02 20:03:15 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Successfully sent a toast message { returnValue: true, toastId: 'HUUE7GGx.756-1585850595424' } 2020-04-02 20:05:27 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as html 2020-04-02 20:05:27 [err] [ManagerDrivers] [webos_plus] [0] _resizeIcon: { Error: Could not find MIME for Buffer at Jimp.parseBitmap (/node_modules/jimp-compact/dist/jimp.js:1:136715) at Jimp.parseBitmap (/node_modules/jimp-compact/dist/jimp.js:1:9529) at new Jimp (/node_modules/jimp-compact/dist/jimp.js:1:8862) at _construct (/node_modules/jimp-compact/dist/jimp.js:1:157616) at /node_modules/jimp-compact/dist/jimp.js:1:15094 at new Promise () at Function.L.read (/node_modules/jimp-compact/dist/jimp.js:1:15052) at Promise (/drivers/webos_plus/device.js:1:142801) at new Promise () at _0xf86d0.(anonymous function)._resizeIcon (/drivers/webos_plus/device.js:1:142665) methodName: 'constructor' } unhandledRejection { Error: Could not find MIME for Buffer at Jimp.parseBitmap (/node_modules/jimp-compact/dist/jimp.js:1:136715) at Jimp.parseBitmap (/node_modules/jimp-compact/dist/jimp.js:1:9529) at new Jimp (/node_modules/jimp-compact/dist/jimp.js:1:8862) at _construct (/node_modules/jimp-compact/dist/jimp.js:1:157616) at /node_modules/jimp-compact/dist/jimp.js:1:15094 at new Promise () at Function.L.read (/node_modules/jimp-compact/dist/jimp.js:1:15052) at Promise (/drivers/webos_plus/device.js:1:142801) at new Promise () at _0xf86d0.(anonymous function)._resizeIcon (/drivers/webos_plus/device.js:1:142665) methodName: 'constructor' }

OllemGit commented 4 years ago

when using this as img url: LG hdmi1 png: http://192.168.2.x:3000/resources/e150e19f45339af2f497a051337040e37108ccda/HDMI_1.png I get a different error message:

2020-04-02 20:09:36 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as tiff 2020-04-02 20:09:39 [log] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast 2020-04-02 20:09:39 [err] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast { returnValue: false, errorCode: -1000, errorText: '161000 bytes icon size greater than maximum allowed 102400 bytes' } 2020-04-02 20:09:39 [err] [ManagerDrivers] [webos_plus] [0] _toastSend: ssap://system.notifications/createToast with result: { returnValue: false, errorCode: -1000, errorText: '161000 bytes icon size greater than maximum allowed 102400 bytes' }

zwaardemaker commented 4 years ago

Will do asap

MaxvandeLaar commented 4 years ago

@OllemGit so you first failed toast shows that your url is not actually returning an image 2020-04-02 20:05:27 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as html the link is probably a redirect to an image. The text after 'message as' displays the mime-type of the image.

The second failed toast shows that the image you are trying to send to the tv is to large. errorText: '161000 bytes icon size greater than maximum allowed 102400 bytes'

If you are indeed using RC7 I have to check something because it should be resized automatically however it seems not to do that for you. A smaller image will probably work you for.

If you test the flow in the online flow editor do you see these error messages?

OllemGit commented 4 years ago

I created a very simple flow using 2 cards: if - when this flow is started then - send a toast message with 1 text: test 2 image: http://192.168.2.x:3000/resources/e150e19f45339af2f497a051337040e37108ccda/HDMI_1.png (this is the hdmi1 image visible when opening the tv set in the app when hdmi1 is selected as input on the tv)

Tested it again with rc7 (double checked) by testing the flow in de https://flow.homey.app/ flow editor but didn't see any error messages there: just an orange lightning indicator in the right corner of the 'than' card showing it has failed...

The error log reads:

_2020-04-03 10:44:57 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as tiff 2020-04-03 10:44:57 [err] [ManagerDrivers] [webos_plus] [0] _resizeIcon: { Error: Could not find MIME for Buffer at Jimp.parseBitmap (/node_modules/jimp-compact/dist/jimp.js:1:136715) at Jimp.parseBitmap (/node_modules/jimp-compact/dist/jimp.js:1:9529) at new Jimp (/node_modules/jimp-compact/dist/jimp.js:1:8862) at _construct (/node_modules/jimp-compact/dist/jimp.js:1:157616) at /node_modules/jimp-compact/dist/jimp.js:1:15094 at new Promise () at Function.L.read (/node_modules/jimp-compact/dist/jimp.js:1:15052) at Promise (/drivers/webos_plus/device.js:1:142801) at new Promise () at _0xf86d0.(anonymous function)._resizeIcon (/drivers/webos_plus/device.js:1:142665) methodName: 'constructor' } unhandledRejection { Error: Could not find MIME for Buffer at Jimp.parseBitmap (/node_modules/jimp-compact/dist/jimp.js:1:136715) at Jimp.parseBitmap (/node_modules/jimp-compact/dist/jimp.js:1:9529) at new Jimp (/node_modules/jimp-compact/dist/jimp.js:1:8862) at _construct (/node_modules/jimp-compact/dist/jimp.js:1:157616) at /node_modules/jimp-compact/dist/jimp.js:1:15094 at new Promise () at Function.L.read (/node_modules/jimp-compact/dist/jimp.js:1:15052) at Promise (/drivers/webos_plus/device.js:1:142801) at new Promise () at _0xf86d0.(anonymous function)._resizeIcon (/drivers/webosplus/device.js:1:142665) methodName: 'constructor' }

MaxvandeLaar commented 4 years ago

@OllemGit, thanks this makes more sense, apparently it's not succeeding in getting the correct mime type from the image and by default it uses tiff. And tiff is not supported when resizing.

Very helpful log! Thanks.

MaxvandeLaar commented 4 years ago

@OllemGit try https://github.com/MaxvandeLaar/homey-webos-plus/releases/tag/v2.0.0-rc8 That should fix your error message

OllemGit commented 4 years ago

Been busy and missed this one... but here the test results!

Test flow (see above) with the png image works now with the remark it just shows 'test' txt and not the 'hdmi' image I would expect. Not clear to me why tiff message would be send since it's a .png?

2020-04-06 08:34:33 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as tiff 2020-04-06 08:34:33 [log] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast 2020-04-06 08:34:33 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Successfully sent a toast message { returnValue: true, toastId: 'eBwCrvSH.20-1586154873851' }

The test to send .jpg image fails: if I replace the local .png file with this .jpg: ( https://s3.eu-central-1.amazonaws.com/plzcdn-zillaimg/800f03d026e08ab0d6c95baebda55ba8.jpg ) it just doens't display with timeout error

2020-04-06 08:50:08 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as jpeg 2020-04-06 08:50:29 [log] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast 2020-04-06 08:50:29 [err] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast Error: timeout at Timeout._onTimeout (/drivers/webos_plus/device.js:1:153325) at ontimeout (timers.js:498:11) at tryOnTimeout (timers.js:323:5) at Timer.listOnTimeout (timers.js:290:5) 2020-04-06 08:50:29 [err] [ManagerDrivers] [webos_plus] [0] _toastSend: ssap://system.notifications/createToast with result: undefined

zwaardemaker commented 4 years ago

@MaxvandeLaar Hi Max, I get the same rmessage with the image

2020-04-06 10:24:08 [log] [ManagerDrivers] [webos_plus] DRIVER TYPE image/jpeg 2020-04-06 10:24:08 [log] [ManagerDrivers] [webos_plus] [0] _toastSend: Send request to create a toast message as jpeg 2020-04-06 10:24:28 [log] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast 2020-04-06 10:24:28 [err] [ManagerDrivers] [webos_plus] [0] _handleResponse: ssap://system.notifications/createToast Error: timeout at Timeout._onTimeout (/drivers/webos_plus/device.js:1:153325) at ontimeout (timers.js:498:11) at tryOnTimeout (timers.js:323:5) at Timer.listOnTimeout (timers.js:290:5) 2020-04-06 10:24:28 [err] [ManagerDrivers] [webos_plus] [0] _toastSend: ssap://system.notifications/createToast with result: undefined

MaxvandeLaar commented 4 years ago

@OllemGit so my code defaults to tiff if the mime type of the image is not detected. Apparently, it can't detect the mime type of your png image thus continues as tiff.

@zwaardemaker @OllemGit, So the tv isn't responding to the API call to create a toast and thus throws an timeout. Since I cannot see the logging from the TV, I can't investigate this any further. If I will have new info, I will let you know here. Thanks again for trying and debugging with me!

zwaardemaker commented 4 years ago

👍

OllemGit commented 4 years ago

Should it work at all the way we try? https://www.webosose.org/docs/reference/ls2-api/com-webos-notification/#createtoast Shows:

iconUrl | Optional | String | The file path of the alert icon. The file path must be local to the device. NOTE​: The icon must be 80 x 80 and in the PNG format.

MaxvandeLaar commented 4 years ago

@OllemGit, I regret to tell you that the documentation you found is sadly not the correct one. WebOS OSE does not fully equal the WebOS on TV's, at least not for the tv's I have tested with.

The closest I got is http://webostv.developer.lge.com/design/webos-tv-system-ui/notifications/

And I quote from that documentation

Because TV users spend most of their time on immersive activities, sending notifications is not allowed on webOS TV. All types of notifications are reserved as system-related events.

So in conclusion to this, I should be glad it is even working for some users. However, the UI does not look the same as on my tv. So this also might be a different OS.

FYI: I am running webos 3.8

OllemGit commented 4 years ago

Similar issues are posted here so It's likely a generic issue with newer LG TV's...

https://community.home-assistant.io/t/lg-webos-notifications-w-icon-problems/142391

https://github.com/home-assistant/core/issues/25935