devpi-ldap fails on Unicode

[fschulze]

See https://bitbucket.org/hpk42/devpi/issue/241/devpi-ldap-fails-on-unicode

[fschulze]

Looking at the traceback again, this seems to be an issue in ldap3 logging code and not in devpi-ldap.

[k-bu]

Hi,

as reported in the devpi issue in the link above, there were 2 issues regarding unicode problems. Meanwhile the one in ldap3 has been fixed, while the one in devpi-ldap hasn't.

Traceback (most recent call last):
  File "/root/conpi/local/lib/python2.7/site-packages/waitress/channel.py", line 338, in service
    task.service()
  File "/root/conpi/local/lib/python2.7/site-packages/waitress/task.py", line 169, in service
    self.execute()
  File "/root/conpi/local/lib/python2.7/site-packages/waitress/task.py", line 399, in execute
    app_iter = self.channel.server.application(env, start_response)
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_server/views.py", line 146, in __call__
    return self.app(environ, start_response)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/router.py", line 270, in __call__
    response = self.execution_policy(environ, self)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/router.py", line 278, in default_execution_policy
    return request.invoke_exception_view(reraise=True)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/view.py", line 755, in invoke_exception_view
    reraise_(*exc_info)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/router.py", line 276, in default_execution_policy
    return router.invoke_request(request)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/router.py", line 249, in invoke_request
    response = handle_request(request)
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_server/views.py", line 170, in request_log_handler
    response = handler(request)
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_server/views.py", line 206, in request_tx_handler
    response = handler(request)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/tweens.py", line 41, in excview_tween
    response = _error_handler(request, exc)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/tweens.py", line 16, in _error_handler
    reraise(*exc_info)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/tweens.py", line 39, in excview_tween
    response = handler(request)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/router.py", line 156, in handle_request
    view_name
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/view.py", line 642, in _call_view
    response = view_callable(context, request)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/viewderivers.py", line 390, in attr_view
    return view(context, request)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/viewderivers.py", line 368, in predicate_wrapper
    return view(context, request)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/viewderivers.py", line 410, in viewresult_to_response
    result = view(context, request)
  File "/root/conpi/local/lib/python2.7/site-packages/pyramid/viewderivers.py", line 124, in _class_requestonly_view
    response = getattr(inst, attr)()
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_server/views.py", line 1101, in login
    proxyauth = self.auth.new_proxy_auth(user, password)
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_server/auth.py", line 87, in new_proxy_auth
    result = self._validate(username, password)
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_server/auth.py", line 49, in _validate
    password=authpassword)
  File "/root/conpi/local/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/root/conpi/local/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/root/conpi/local/lib/python2.7/site-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/root/conpi/local/lib/python2.7/site-packages/pluggy/callers.py", line 201, in _multicall
    return outcome.get_result()
  File "/root/conpi/local/lib/python2.7/site-packages/pluggy/callers.py", line 77, in get_result
    _reraise(*ex)  # noqa
  File "/root/conpi/local/lib/python2.7/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_ldap/main.py", line 277, in devpiserver_auth_user
    return ldap.validate(username, password)
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_ldap/main.py", line 255, in validate
    groups = self._search(conn, config, username=username, userdn=userdn)
  File "/root/conpi/local/lib/python2.7/site-packages/devpi_ldap/main.py", line 174, in _search
    search_filter = config['filter'].format(**kw)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 10: ordinal not in range(128)

But these patches mentioned in the link above still work:

2) devpi_ldap.main.LDAP._search:
old: config['base'], search_filter,
new: config['base'], search_filter.decode("utf8"),
3) devpi_ldap.main.LDAP._userdn:
old: result = self._search(None, self['user_search'], username=username)
new: result = self._search(None, self['user_search'], username=username.encode("utf8"))
4) devpi_ldap.main.LDAP.validate:
old: groups = self._search(conn, config, username=username, userdn=userdn)
new: groups = self._search(conn, config, username=username, userdn=userdn.encode("utf8"))

Used versions:

devpi-ldap (1.2.2)
devpi-server (4.7.1)
ldap3 (2.5.1)

Regards

[k-bu]

Only after seeing the failed tests of the pull request above with Python 3 I realized that everything works fine with Python 3. I've tested it right now. Thus there should be a Python 2 only fix.