UnicodeEncodeError in unquoting the path

peterbe commented 5 years ago

https://sentry.prod.mozaws.net/operations/kinto-webextensions-prod/issues/4723172/?query=is:unresolved

UnicodeEncodeError: 'ascii' codec can't encode characters in position 144-147: ordinal not in range(128)
  File "pyramid/tweens.py", line 39, in excview_tween
    response = handler(request)
  File "kinto/core/events.py", line 76, in tween
    response = handler(request)
  File "pyramid/router.py", line 156, in handle_request
    view_name
  File "pyramid/view.py", line 642, in _call_view
    response = view_callable(context, request)
  File "pyramid/config/views.py", line 181, in __call__
    return view(context, request)
  File "pyramid/viewderivers.py", line 390, in attr_view
    return view(context, request)
  File "pyramid/viewderivers.py", line 368, in predicate_wrapper
    return view(context, request)
  File "pyramid/viewderivers.py", line 439, in rendered_view
    result = view(context, request)
  File "pyramid/viewderivers.py", line 148, in _requestonly_view
    response = view(request)
  File "cornice/service.py", line 502, in wrapper
    response = view_(request)
  File "kinto/core/views/batch.py", line 139, in post_batch
    subrequest = build_request(request, subrequest_spec)
  File "kinto/core/utils.py", line 370, in build_request
    method=method)
  File "webob/request.py", line 1346, in blank
    env = environ_from_url(path)
  File "webob/request.py", line 1443, in environ_from_url
    path_info = url_unquote(path)
  File "webob/compat.py", line 103, in url_unquote
    return unquote(s.encode('ascii')).decode('latin-1')

According to Sentry the s variable is '/v1/buckets/default/collections/ext-8f641XJPcYKMevbo_IHSjtztGey341gtEiw9ABlao6s/records/id-1yESnI5FCxK_U2TXXzeuYA7bl-sOkttf-9Kcs2yHa-c7hdfrm4fw0ÃÂepzt2lrn49'

leplatrem commented 5 years ago

Those were @Natim favorites :D

This happens in the batch endpoint which is a bit specific. Some more safety checks might be necessary.

Otherwise, this should help greatly https://github.com/Pylons/webob/pull/390

Natim commented 5 years ago

I've got to say that thinking of a path as ascii in 2018 seems rather strange to me.

digitalresistor commented 5 years ago

@Natim PEP0333 :-/

Natim commented 5 years ago

It helps thanks!

digitalresistor commented 5 years ago

I'm working (slowly in my spare time) on WebOb 2.0 which should make it easier to catch the unicode decode errors by turning them into a webob specific exception that can easily be caught and dealt with.

Kinto / kinto

UnicodeEncodeError in unquoting the path #1894