search

michaelkourlas / voipms-sms-client

Popular Android messaging app for VoIP.ms, a Canadian VoIP provider
https://play.google.com/store/apps/details?id=net.kourlas.voipms_sms
Apache License 2.0
221 stars 52 forks source link

getMMS API is returning HTML/XML character entity references instead of valid JSON #251

Closed yulman19 closed 1 year ago

yulman19 commented 1 year ago

Letters like this "é" come out as "&eacute" in text messages

michaelkourlas commented 1 year ago

I can pretty much guarantee that this is not a problem with the app, which is just handing off standard UTF-8 to the API.

From the VoIP.ms docs :

We cannot guarantee that accents or special characters including non-Latin letters will be properly delivered.

Are you able to send the same message using the same API call with the accent preserved?

michaelkourlas commented 1 year ago

Or are you talking about characters in incoming messages? The app just displays exactly what it gets from the API.

baldurmen commented 1 year ago

I have the same issue. Here are two screenshots of the same SMS messages, the first in the webapp (sms.voip.ms) and the second in the app (version v0.6.24, f-droid). This was not an issue in v0.6.23.

Web interface:

sms_web_interface

App:

sms_app_06 24

michaelkourlas commented 1 year ago

0.6.24 uses the getMMS API call, whereas 0.6.23 uses getSMS -- maybe there's a difference between the two APIs?

baldurmen commented 1 year ago

Probably? Those are HTML entities and maybe the result of the getMMS [message] output is intended to be parsed via some HTML->UTF-8 function? The API doesn't mention anything, but I guess this might be the result of voip.ms interpreting the MIME of the original MMS and re-encoding it to something consistent (since we don't have access to that information via the API).

I did some further tests and I can write messages to someone with non-ascii chars without issues. They are displayed properly in the app and also received properly. So this only seems to affect incoming messages and not outcoming ones.

  1. I send un message avec des accents: é à ô
  2. The contact receives un message avec des accents: é à ô
  3. They send back un message avec des accents: é à ô
  4. I get un message avec des accents: é à ô

screenchot

michaelkourlas commented 1 year ago

I've confirmed the APIs are different. This from getSMS:

{"id":"66627691","date":"2023-06-09 21:48:25","type":"1","did":"<redacted>","contact":"<redacted>","message":"\u00c9"}

becomes this in getMMS:

{"id":"66627691","date":"2023-06-0921:48:25","type":"1","did":"<redacted>","contact":"<redacted>","message":"&Eacute;","col_media1":"","col_media2":"","col_media3":"","media":[]}
michaelkourlas commented 1 year ago

I'm going to leave this open for now and open a ticket.

baldurmen commented 1 year ago

I don't know if Voip.MS support replied to your ticket, but it seems that since 2023-06-27 (or at least, that when I saw the change), this problem is not longer happening and I get proper characters instead of those HTML entities.

I guess something changed on their side?

michaelkourlas commented 1 year ago

Yes, a rep got back to me and informed me the issue has been fixed.