`posts://` Encoding Error in notifications

[Emporea]

I was tracking a website that made a change using a smiley ('ツ'). I saw that changedetection recognized the change, but it didnt produce a notification via posts://.

Logs said this:

2024/04/16 15:55:31,000 - SENDING - null
2024-04-16 15:55:31,350 - DEBUG - <class 'apprise.decorators.CustomNotifyPlugin.CustomNotifyPlugin.instantiate_plugin.<locals>.CustomNotifyPluginWrapper'> Exception: 'latin-1' codec can't encode character '\u30c4' in position 54: Body ('ツ') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.
2024-04-16 15:55:31,350 - WARNING - An exception occured sending a Custom - posts notification.
2024-04-16 15:55:31,288 - DEBUG - Loaded Custom - posts URL: posts://

Is there something I can do or does this need to be done in the code of the NotifyPlugin?

Also i am wondering about the two different date formats, I dont know why the plugin uses a different one.

[caronc]

Let me look into this when I get home this week. Imo all encodings should beutf8. I'm not sure why it's trying to encode your post with latin-1

[caronc]

So I looked further into this and I think the issue may be on @Emporea's end. The CustomPlugin is a library i wrote so people could import and work with their own modules they write. This error is actually coming from within the custom plugin written (and not Apprise). Apprise is just gracefully catching the error, displaying it, and then moving along to the next notification (if there is one).

@Emporea perhaps you can explain a bit more on the plugin you're using? The advice of the error is pretty good too. Probably just adding body.encode('utf-8') will fix your issue. I'll still have a look into my source; i'm confident i'm handing the body along in unicode, but i'll still confirm this.

[Emporea]

I haven't installed any plugins. I am using the default docker image. Then I just used the posts:// notification command, as desribed and suggested in the notification settings.

So what exactly do you mean by saying the issue may be on @Emporea's end?

[caronc]

Strange, the error implies you're environment has a @notify custom plugin. ~But i could be wrong~.

Edit: There is definitely something that is customized because even your error message says it's trying to initialize a posts:// which does not exist under Apprise by itself.

[Emporea]

It's not part of apprise but it is part of change detection.

Docker image: ghcr.io/dgtlmoon/changedetection.io

Thats the official docker image, right?

[caronc]

@dgtlmoon maybe can add to this. It looks like under the hood he perhaps created some custom hooks. The hooks appear to be re-encoding the utf-8 data it's provided. I'm not sure what the logic is that occurs inside these.

[dgtlmoon]

@caronc you are correct :) thanks again for your input, yes thats our little custom extension

[maksm]

Greetings, can confirm this error happened on my 'vanilla' deployment, I am using post to a Gotify notification service which failed in one instance which contained a non-latin character:

2024-04-30 00:41:50,652 - DEBUG - <class 'apprise.decorators.CustomNotifyPlugin.CustomNotifyPlugin.instantiate_plugin.<locals>.CustomNotifyPluginWrapper'> Exception: 'latin-1' codec can't encode character '\u0161' in position 75: Body ('š') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.

[zxkmm]

hello, can confirm this exist when chinese char in the body, and using Bark with gets hook

[Tschrock]

If you are using JSON for your posts:// - Make sure you are using | tojson when building your json message, this should encode anything non-ascii and bypass this error. For example, it will turn the smiley ツ into \u30c4

Some examples here - this one does not do any encoding and you get the error (It will also have issues if your diff has a " double quote):

{ "message": "{{ diff }}" }

But with this one, everything is encoded and you do not get the error:

{ "message": {{ diff | tojson }} }

[elreydetoda]

So, I've very new to changedetection, but it looks like the issue (if it wasn't being encoded) would be here (specifically line number 110):

https://github.com/dgtlmoon/changedetection.io/blob/755cba33de1f702d4b8fe91e8a8f7eb29bb8af53/changedetectionio/notification.py#L108-L113

Would doing something like what was suggested here https://github.com/dgtlmoon/changedetection.io/issues/1315#issuecomment-2180299549 be an option for ensuring the encoding is forced?

[MFYDev]

Just met this issue as well, cannot send out the proper posts notification to my webhook at all. Same log is showing it is using latin-1. I have a workflow based on the webhook triggering. Since this error the whole workflow cannot work. Will hugely appreciate it if @dgtlmoon can fix this issue. Thanks!

[hz153jht]

codezip.zip

Replace the attached requests.py with changedetectionio/ content_fetchers/requests.py and notification.py with changedetectionio/notification.py. Then restart the docker container again. My application scenario is to send changes monitored by changedetection via gotify. Only for version 0.46.01