zopefoundation / zope.publisher

Map requests from HTTP/WebDAV clients, web browsers, XML-RPC and FTP clients onto Python objects
Other
3 stars 13 forks source link

Tests fail on Python 3.3 #3

Closed mgedmin closed 9 years ago

mgedmin commented 10 years ago

tox -e py33:

/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/interface/common/tests/basemapping.py:22: DeprecationWarning: Please use assertTrue instead.
  self.failUnless(key in inst)
......................................EE................................................................................................EE................................/usr/lib/python3.3/unittest/case.py:384: ResourceWarning: unclosed file <_io.BufferedRandom name=4>
  function()
/usr/lib/python3.3/unittest/case.py:384: ResourceWarning: unclosed file <_io.BufferedRandom name=5>
  function()
/usr/lib/python3.3/unittest/case.py:384: ResourceWarning: unclosed file <_io.BufferedRandom name=6>
  function()
..............................................................................
======================================================================
ERROR: testCookieErrorToLog (zope.publisher.tests.test_browserrequest.BrowserTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_http.py", line 478, in testCookieErrorToLog
    req = self._createRequest(extra_env=cookies)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_browserrequest.py", line 146, in _createRequest
    request = TestBrowserRequest(instream, env)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_browserrequest.py", line 79, in __init__
    BrowserRequest.__init__(self, *args, **kw)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/browser.py", line 242, in __init__
    super(BrowserRequest, self).__init__(body_instream, environ, response)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 388, in __init__
    self.__setupCookies()
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 496, in __setupCookies
    self._parseCookies(cookie_header, self._cookies)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 474, in _parseCookies
    c = LenientCookie(text)
  File "/usr/lib/python3.3/http/cookies.py", line 481, in __init__
    self.load(input)
  File "/usr/lib/python3.3/http/cookies.py", line 526, in load
    self.__parse_string(rawdata)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 222, in _BaseCookie__parse_string
    self._BaseCookie__ParseString(str, patt)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 213, in _BaseCookie__ParseString
    rval, cval = self.value_decode(V)
  File "/usr/lib/python3.3/http/cookies.py", line 576, in value_decode
    return _unquote(val), val
  File "/usr/lib/python3.3/http/cookies.py", line 247, in _unquote
    if len(str) < 2:
TypeError: object of type 'NoneType' has no len()

======================================================================
ERROR: testCookies (zope.publisher.tests.test_browserrequest.BrowserTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_http.py", line 455, in testCookies
    req = self._createRequest(extra_env=cookies)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_browserrequest.py", line 146, in _createRequest
    request = TestBrowserRequest(instream, env)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_browserrequest.py", line 79, in __init__
    BrowserRequest.__init__(self, *args, **kw)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/browser.py", line 242, in __init__
    super(BrowserRequest, self).__init__(body_instream, environ, response)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 388, in __init__
    self.__setupCookies()
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 496, in __setupCookies
    self._parseCookies(cookie_header, self._cookies)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 474, in _parseCookies
    c = LenientCookie(text)
  File "/usr/lib/python3.3/http/cookies.py", line 481, in __init__
    self.load(input)
  File "/usr/lib/python3.3/http/cookies.py", line 526, in load
    self.__parse_string(rawdata)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 222, in _BaseCookie__parse_string
    self._BaseCookie__ParseString(str, patt)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 213, in _BaseCookie__ParseString
    rval, cval = self.value_decode(V)
  File "/usr/lib/python3.3/http/cookies.py", line 576, in value_decode
    return _unquote(val), val
  File "/usr/lib/python3.3/http/cookies.py", line 247, in _unquote
    if len(str) < 2:
TypeError: object of type 'NoneType' has no len()

======================================================================
ERROR: testCookieErrorToLog (zope.publisher.tests.test_http.ConcreteHTTPTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_http.py", line 478, in testCookieErrorToLog
    req = self._createRequest(extra_env=cookies)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_http.py", line 221, in _createRequest
    request = HTTPRequest(instream, env)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 388, in __init__
    self.__setupCookies()
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 496, in __setupCookies
    self._parseCookies(cookie_header, self._cookies)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 474, in _parseCookies
    c = LenientCookie(text)
  File "/usr/lib/python3.3/http/cookies.py", line 481, in __init__
    self.load(input)
  File "/usr/lib/python3.3/http/cookies.py", line 526, in load
    self.__parse_string(rawdata)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 222, in _BaseCookie__parse_string
    self._BaseCookie__ParseString(str, patt)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 213, in _BaseCookie__ParseString
    rval, cval = self.value_decode(V)
  File "/usr/lib/python3.3/http/cookies.py", line 576, in value_decode
    return _unquote(val), val
  File "/usr/lib/python3.3/http/cookies.py", line 247, in _unquote
    if len(str) < 2:
TypeError: object of type 'NoneType' has no len()

======================================================================
ERROR: testCookies (zope.publisher.tests.test_http.ConcreteHTTPTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_http.py", line 455, in testCookies
    req = self._createRequest(extra_env=cookies)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/tests/test_http.py", line 221, in _createRequest
    request = HTTPRequest(instream, env)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 388, in __init__
    self.__setupCookies()
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 496, in __setupCookies
    self._parseCookies(cookie_header, self._cookies)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 474, in _parseCookies
    c = LenientCookie(text)
  File "/usr/lib/python3.3/http/cookies.py", line 481, in __init__
    self.load(input)
  File "/usr/lib/python3.3/http/cookies.py", line 526, in load
    self.__parse_string(rawdata)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 222, in _BaseCookie__parse_string
    self._BaseCookie__ParseString(str, patt)
  File "/home/mg/src/new-zope-order/zope.publisher/.tox/py33/lib/python3.3/site-packages/zope/publisher/http.py", line 213, in _BaseCookie__ParseString
    rval, cval = self.value_decode(V)
  File "/usr/lib/python3.3/http/cookies.py", line 576, in value_decode
    return _unquote(val), val
  File "/usr/lib/python3.3/http/cookies.py", line 247, in _unquote
    if len(str) < 2:
TypeError: object of type 'NoneType' has no len()

----------------------------------------------------------------------
Ran 248 tests in 0.291s

FAILED (errors=4)
jamur2 commented 9 years ago

This is partially the fault of https://github.com/zopefoundation/zope.publisher/pull/2 which was a hack that is way too tightly coupled to the stdlib Cookie internals.

Even with that change backed out (or updated to work with the latest Python releases), the stricter cookie implementation changed behavior between 2.7.8 and 2.7.9 (and presumably between 3.3.5 and 3.3.6 as well), so existing tests will probably need to be updated with a stricter cookie string (and cookies that were accepted even before LenientCookie would stop being accepted--although that may be considered a security feature):

Python 2.7.8:

>>> import Cookie
>>> Cookie.SimpleCookie('foo=bar; path=/; spam="eggs", this="Should be fine"')
<SimpleCookie: foo='bar' spam='eggs' this='Should be fine'>

Python 2.7.9

>>> import Cookie
>>> Cookie.SimpleCookie('foo=bar; path=/; spam="eggs", this="Should be fine"')
<SimpleCookie: foo='bar'>

The LenientCookie was an attempt to follow "Practicality Beats Purity" and "Be conservative in what you do, be liberal in what you accept from others", because we were seeing bogus (but not malicious) cookies being set by 3rd parties that were breaking sessions on some sites by invalidating all cookies. If this is a real security concern (as https://hg.python.org/cpython/rev/9e765e65e5cb makes it look), maybe it's not worth trying to be accepting of invalid cookies (although one might consider setting an invalid cookie a denial of service attack against browser sessions I guess).

I dunno, I won't be too sad if LenientCookie got ripped out and we updated our tests to include only valid cookies as defined by the latest Python Cookie module implementation. @freddrake?

mgedmin commented 9 years ago

Looks like https://bugs.launchpad.net/zope.publisher/+bug/1404582 is the same bug

tseaver commented 9 years ago

Thanks, @jamur2, for figuring out how to fix the issue.