Issue with UTF-8 caracters as parameters with python 3

[titilambert]

Hello !

I have an issue when I'm trying to call a remote function with a UTF-8 caracter. Here my example

Server

from wampy.peers.clients import Client
from wampy.roles.callee import callee

class BinaryNumberService(Client):

    @callee
    def get_binary_number(self, text):
        print(text)

toto  = BinaryNumberService()

toto.start()

try:
    import time
    time.sleep(9999)
except:
    print('stopping')
toto.stop()

Client

from wampy.peers.clients import Client

with Client(url="ws://localhost:8080") as client:
            result = client.rpc.get_binary_number(text='100éfa')

And the error:

python client.py 
Traceback (most recent call last):
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/transports/websocket/frames.py", line 239, in __init__
    self.payload = json.loads(str(self.body.decode()))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xea in position 1: invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/env3.6/lib/python3.6/site-packages/eventlet-0.21.0-py3.6.egg/eventlet/hubs/poll.py", line 114, in wait
    listener.cb(fileno)
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/env3.6/lib/python3.6/site-packages/eventlet-0.21.0-py3.6.egg/eventlet/greenthread.py", line 218, in main
    result = function(*args, **kwargs)
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/session.py", line 159, in connection_handler
    frame = connection.receive()
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/transports/websocket/connection.py", line 83, in receive
    frame = ServerFrame(received_bytes)
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/transports/websocket/frames.py", line 242, in __init__
    'Failed to load JSON object from: "%s"', self.body
wampy.errors.WebsocktProtocolError: ('Failed to load JSON object from: "%s"', bytearray(b"\x03\xeaWAMP Protocol Error (invalid serialization of WAMP message (Expecting \',\' delimiter: line 1 column 56 (char 55)))"))
Removing descriptor: 3
Traceback (most recent call last):
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/session.py", line 115, in recv_message
    message = self._wait_for_message(timeout)
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/session.py", line 184, in _wait_for_message
    eventlet.sleep()
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/env3.6/lib/python3.6/site-packages/eventlet-0.21.0-py3.6.egg/eventlet/greenthread.py", line 35, in sleep
    hub.switch()
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/env3.6/lib/python3.6/site-packages/eventlet-0.21.0-py3.6.egg/eventlet/hubs/hub.py", line 295, in switch
    return self.greenlet.switch()
eventlet.timeout.Timeout: 5 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "client.py", line 6, in <module>
    result = client.rpc.get_binary_number(text='100éfa')
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/roles/caller.py", line 63, in wrapper
    response = self.client.make_rpc(message)
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/peers/clients.py", line 211, in make_rpc
    response = self.session.recv_message()
  File "/home/tcohen/perso/gits/github.com/noisyboiler/wampy/wampy/session.py", line 118, in recv_message
    "no message returned (timed-out in {})".format(timeout)
wampy.errors.WampProtocolError: no message returned (timed-out in 5)

[noisyboiler]

thanks @titilambert looks like a bug! I'll get this looked at over the coming days. You're welcome to raise a PR too if you'd like to help out? otherwise I'll sort this out next weekend. Thanks for bringing this to my attention.

[titilambert]

@noisyboiler I has been trying to fix it since yesterday, without success :( I will continue to try, I hope you will be quicker than me :)

[oddjobz]

This vaguely rings a bell, can someone with the problem humour me and tell me what this comes back with?

python -c 'import sys; print(sys.getdefaultencoding())'

[titilambert]

@oddjobz

$ python -c 'import sys; print(sys.getdefaultencoding())'
ascii
$ python3 -c 'import sys; print(sys.getdefaultencoding())'
utf-8

[oddjobz]

Ok, are you mixing v2/v3 with client / server? If so, that may be the issue, setting both to utf-8 could well sort the problem. Number of ways of doing this, in the past I've change the default encoding in "sitecustomize.py", see this article on stackoverflow

[titilambert]

@oddjobz Sorry, my answer wasn't clear, I'm using python 3 ONLY. I just wanted to give the difference with python 2

[oddjobz]

Ok, but the issue "looks" like it's trying to decode something in utf-8 that's been encoded in something other than utf-8. (historically this problem is typical of swapping data between v2 and v3 instances) I don't suppose you're specifically changing the encoding anywhere in your code?

[noisyboiler]

@titilambert can you try substituting one of wampy's existing test case calls with the text text='100éfa' to see if this is a wampy problem or not? I'd like coverage of this regardless so it's something I'll try asap, but you're welcome to raise a PR that adds the coverage with a new test for this - else proves there is a bug! thanks

[titilambert]

@noisyboiler I just saw, you did it here #44. Did you reproduce the bug ?

[noisyboiler]

yes! sorry, there is a bug in payload length with characters beyond standard ascii length.

[noisyboiler]

this was a combination of issues

• sys.getdefaultencoding() being ascii on our machines :(
• flakey serialisation that can return different things based on the above and the python version
• giant bug in __len__ implementation
• random decodes that made things work for ascii but were not properly understood

[noisyboiler]

fixed by https://github.com/noisyboiler/wampy/pull/44