quoted urls UnicodeDecodeError

[laurentpayot]

I use quoted URLs such as /fr/livres/cr%C3%A9ation/ (for /fr/livres/création/). get() is working fine with these urls but once I have the response if I try to use click() or submit() on it I get the following error:

/usr/lib/python3.3/http/cookiejar.py:1588: UserWarning: http.cookiejar bug!
Traceback (most recent call last):
  File "/usr/lib/python3.3/http/cookiejar.py", line 1586, in make_cookies
    parse_ns_headers(ns_hdrs), request)
  File "/usr/lib/python3.3/http/cookiejar.py", line 1543, in _cookies_from_attrs_set
    cookie = self._cookie_from_cookie_tuple(tup, request)
  File "/usr/lib/python3.3/http/cookiejar.py", line 1492, in _cookie_from_cookie_tuple
    req_host, erhn = eff_request_host(request)
  File "/usr/lib/python3.3/http/cookiejar.py", line 612, in eff_request_host
    erhn = req_host = request_host(request)
  File "/usr/lib/python3.3/http/cookiejar.py", line 597, in request_host
    url = request.get_full_url()
  File "/usr/local/lib/python3.3/dist-packages/WebTest-2.0.8.dev0-py3.3.egg/webtest/utils.py", line 116, in get_full_url
    return self._request.url
  File "/usr/local/lib/python3.3/dist-packages/WebOb-1.2.3-py3.3.egg/webob/request.py", line 505, in url
    url = self.path_url
  File "/usr/local/lib/python3.3/dist-packages/WebOb-1.2.3-py3.3.egg/webob/request.py", line 477, in path_url
    bpath_info = bytes_(self.path_info, self.url_encoding)
  File "/usr/local/lib/python3.3/dist-packages/WebOb-1.2.3-py3.3.egg/webob/descriptors.py", line 68, in fget
    return req.encget(key, encattr=encattr)
  File "/usr/local/lib/python3.3/dist-packages/WebOb-1.2.3-py3.3.egg/webob/request.py", line 167, in encget
    return bytes_(val, 'latin-1').decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 5: invalid continuation byte

  _warn_unhandled_exception()

I'm using Ubuntu 13.04 64 bit and Python 3.3.1 with the last commit of Webtest.

[gawel]

I think it's more a problem in WebOb / WSGI PEP. Every url should be latin1 encoded and the real url encoding is lost during the process (see hard coded latin-1 encoding in the traceback...).

Not sure that we'll be able to fix that one day...

[gawel]

Also, looks like it's related to https://github.com/Pylons/webtest/issues/1

[NomadDemon]

i had this problem today i had to "hack" webob to use UTF, to allow send url = reverse('someurl', kwargs={'name': 'żółć'}) how about some hack/parameter for this? to allow UTF in urls?

[rjsparks]

@gawel - bump?

We are going to pull WebTest out of one of our projects over this for now, which makes me sad.

This URL really should be useable through this test framework: /community/personal/%C3%89ric%20Vyncke/manage/ But by the time it gets to return bytes_(val, 'latin-1').decode(encoding) in webop.request.encget, the %C3%89 has been unescaped and val has an É in it.

[gawel]

@rjsparks as you noticed the "problem" is in webob, not webtest.

[rjsparks]

Why the quotes around "problem"?

@bertjwregeer - I don't see a relevant issue at webob on a quick read. Are you already aware of this behavior?

[laurentpayot]

I’m unsubscribing from this 10 years old thread because I moved from Python to JS/TS and Elm a loooong time ago. :sauropod: I will let the repo owner close this issue when fixed… or not :wink: