search

RestComm / visual-designer

Restcomm Visual Designer
http://www.restcomm.com/
GNU Affero General Public License v3.0
4 stars 15 forks source link

Verify proper use of emoji in utf-8 #338

Closed otsakir closed 6 years ago

otsakir commented 6 years ago

Make sure emoji characters are properly transmitted designer and restcomm.

otsakir commented 6 years ago

First things first. UTF-8 is capable of encoding all possible Unicode characters according to wikipedia: https://en.wikipedia.org/wiki/UTF-8

Thus, making sure that such characters are properly stored in Designer projects and then returned as valid UTF-8 rcml code to Restcomm should be enough.

otsakir commented 6 years ago

https://unicode.org/emoji/charts/full-emoji-list.html

otsakir commented 6 years ago

I checked actual binary representation of emoji-containing Say/SMS elements. Both, filesystem storage and RCML code produced contain valid UTF-8 encoded byte sequences.

😀 --> F0 9F 98 80

otsakir commented 6 years ago

I also ran some tests involving restcomm and Olympus. I created a Designer SMS application that responds back to the origin of the message it receives.

  1. Sending back an SMS to Olymups with emoji that is hardcoded in a Designer application works.
  2. Sending an SMS that contains emoji from Olympus to Designer seems problematic. The request for RCML does not reach Designer initially. Instead one needs to send yet another message without emoji to get a response back that contains some sort of garbage too:

sent:

 hhhh😀
 asdfasdf

received:

 welcome. echo: hhhh😀ME

Also, from wireshark i got:

POST /restcomm-rvd/services/apps/AP0a8b77f1b45a436b825fc05942386187/controller HTTP/1.1
http.protocol.content-charset: UTF-8
Content-Length: 130
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Host: 192.168.2.2:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.2 (Java/1.7.0_80)
Accept-Encoding: gzip,deflate

SmsSid=SMed0c870ad02c43178af2572ba1c5cc7b&AccountSid=ACae6e420f425248d6a26948c17a9e2acf&From=alice&To=4455&Body=hhhh%F0%9F%98%80MEHTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Mon, 30 Apr 2018 10:04:44 GMT

54
<Response>
  <Sms to="alice" from="4455">welcome. echo: hhhh....ME</Sms>
</Response>
0

I sent two messages. The first did not result in a request for RCML. After the second a request was generated with both messages and for some reason a 'ME' part is added to the end as seen in the POST body above..

otsakir commented 6 years ago

The bug described above was in Olympus and has been fixed now.