test failures for 18.3.1 release

[dol-sen]

I am maintaining the Gentoo package, while working on bumping this latest release:

Test phase: dev-python/autobahn-18.3.1

• python2_7: running distutils-r1_run_phase python_test =============================================================================== test session starts =============================================================================== platform linux2 -- Python 2.7.14, pytest-3.0.3, py-1.4.30, pluggy-0.4.0 -- /usr/bin/python2.7 cachedir: .cache rootdir: /var/tmp/portage/dev-python/autobahn-18.3.1/work/autobahn-18.3.1-python2_7/lib, inifile: plugins: cov-2.3.1, hypothesis-3.6.0, betamax-20180307 collecting ... collected 163 items

==================================================================================== FAILURES ===================================================================================== ___________________________________________________________________________ TestAuth.test_authenticator ___________________________________________________________________________ self = def test_authenticator(self): authenticator = create_authenticator( u"cryptosign", authid="someone", privkey=self.privkey_hex, ) session = Mock() session._transport.get_channel_id = Mock(return_value=self.channel_id) challenge = types.Challenge(u"cryptosign", dict(challenge="ff" * 32)) reply = authenticator.on_challenge(session, challenge) self.assertEqual( reply.result, > u'9b6f41540c9b95b4b7b281c3042fa9c54cef43c842d62ea3fd6030fcb66e70b3e80d49d44c29d1635da9348d02ec93f3ed1ef227dfb59a07b580095c2b82f80f9d16ca518aa0c2b707f2b2a609edeca73bca8dd59817a633f35574ac6fd80d00', ) E AssertionError: > != u'9b6f41540c9b95b4b7b281c3042fa9c54cef43c[149 chars]0d00' autobahn/wamp/test/test_cryptosign.py:87: AssertionError _______________________________________________________________________________ TestAuth.test_valid _______________________________________________________________________________ self = def test_valid(self): session = Mock() session._transport.get_channel_id = Mock(return_value=self.channel_id) challenge = types.Challenge(u"ticket", dict(challenge="ff" * 32)) signed = self.key.sign_challenge(session, challenge) self.assertEqual( u'9b6f41540c9b95b4b7b281c3042fa9c54cef43c842d62ea3fd6030fcb66e70b3e80d49d44c29d1635da9348d02ec93f3ed1ef227dfb59a07b580095c2b82f80f9d16ca518aa0c2b707f2b2a609edeca73bca8dd59817a633f35574ac6fd80d00', > signed.result, ) E AssertionError: u'9b6f41540c9b95b4b7b281c3042fa9c54cef43c[149 chars]0d00' != > autobahn/wamp/test/test_cryptosign.py:72: AssertionError ====================================================================== 2 failed, 161 passed in 1.17 seconds =======================================================================

[oberstet]

What version of txaio are you using?

[dol-sen]

I have txaio-2.8.1 currently installed, I'll retest with 2.9.0, but I see the dep is still at >=2.7.0

[dol-sen]

I am getting the same test failures with txaio-2.7.1, 2.8.1, 2.9.0

[oberstet]

I can repeat that locally:

(cpy362_9) oberstet@thinkpad-t430s:~/scm/crossbario/autobahn-python$ make test_asyncio 
USE_ASYNCIO=1 python -m pytest -rsx autobahn
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.6.2, pytest-3.2.5, py-1.5.2, pluggy-0.4.0
rootdir: /home/oberstet/scm/crossbario/autobahn-python, inifile: setup.cfg
plugins: asyncio-0.5.0, aiohttp-0.3.0
collected 173 items                                                                                                                                                        

autobahn/asyncio/test/test_asyncio_rawsocket.py ........
autobahn/asyncio/test/test_asyncio_websocket.py x.
autobahn/rawsocket/test/test_rawsocket_url.py .................
autobahn/test/test_util.py ..
autobahn/wamp/test/test_auth.py .........
autobahn/wamp/test/test_cryptosign.py FF...
autobahn/wamp/test/test_exception.py ..
autobahn/wamp/test/test_message.py ...............................................................
autobahn/wamp/test/test_protocol_peer.py ..
autobahn/wamp/test/test_runner.py .....
autobahn/wamp/test/test_serializer.py ....
autobahn/wamp/test/test_uri_pattern.py ................
autobahn/websocket/test/test_protocol.py .........
autobahn/websocket/test/test_websocket_url.py .............................
========================================================================= short test summary info =========================================================================
XFAIL autobahn/asyncio/test/test_asyncio_websocket.py::Test::test_async_on_connect_server

================================================================================ FAILURES =================================================================================
_______________________________________________________________________ TestAuth.test_authenticator _______________________________________________________________________

self = <autobahn.wamp.test.test_cryptosign.TestAuth testMethod=test_authenticator>

    def test_authenticator(self):
        authenticator = create_authenticator(
            u"cryptosign",
            authid="someone",
            privkey=self.privkey_hex,
        )
        session = Mock()
        session._transport.get_channel_id = Mock(return_value=self.channel_id)
        challenge = types.Challenge(u"cryptosign", dict(challenge="ff" * 32))
        reply = authenticator.on_challenge(session, challenge)
        self.assertEqual(
            reply.result,
>           u'9b6f41540c9b95b4b7b281c3042fa9c54cef43c842d62ea3fd6030fcb66e70b3e80d49d44c29d1635da9348d02ec93f3ed1ef227dfb59a07b580095c2b82f80f9d16ca518aa0c2b707f2b2a609edeca73bca8dd59817a633f35574ac6fd80d00',
        )
E       AssertionError: <built-in method result of _asyncio.Futur[22 chars]f898> != '9b6f41540c9b95b4b7b281c3042fa9c54cef43c8[148 chars]0d00'

autobahn/wamp/test/test_cryptosign.py:87: AssertionError
___________________________________________________________________________ TestAuth.test_valid ___________________________________________________________________________

self = <autobahn.wamp.test.test_cryptosign.TestAuth testMethod=test_valid>

    def test_valid(self):
        session = Mock()
        session._transport.get_channel_id = Mock(return_value=self.channel_id)
        challenge = types.Challenge(u"ticket", dict(challenge="ff" * 32))
        signed = self.key.sign_challenge(session, challenge)
        self.assertEqual(
            u'9b6f41540c9b95b4b7b281c3042fa9c54cef43c842d62ea3fd6030fcb66e70b3e80d49d44c29d1635da9348d02ec93f3ed1ef227dfb59a07b580095c2b82f80f9d16ca518aa0c2b707f2b2a609edeca73bca8dd59817a633f35574ac6fd80d00',
>           signed.result,
        )
E       AssertionError: '9b6f41540c9b95b4b7b281c3042fa9c54cef43c8[148 chars]0d00' != <built-in method result of _asyncio.Futur[22 chars]f5f8>

autobahn/wamp/test/test_cryptosign.py:72: AssertionError
============================================================= 2 failed, 170 passed, 1 xfailed in 1.11 seconds =============================================================
Makefile:94: die Regel für Ziel „test_asyncio“ scheiterte
make: *** [test_asyncio] Fehler 1
(cpy362_9) oberstet@thinkpad-t430s:~/scm/crossbario/autobahn-python$ 

the CI has these tests skipped sadly:

• https://travis-ci.org/crossbario/autobahn-python
• https://travis-ci.org/crossbario/autobahn-python/jobs/349274788#L524

[oberstet]

oh, I've been doing above test on a branch https://github.com/crossbario/autobahn-python/pull/955

[oberstet]

@dol-sen ok, so I've adjusted our CI so that it reproduces the problem (we missed the necessary deps in the CI runs, so the unit tests that fail for you where skipped in our CI - fixed now): https://travis-ci.org/crossbario/autobahn-python/builds/351181363

I will now actually analyze the issue;) I let you know ..

[oberstet]

this was actually a bug in the unit test (not in the tested code): https://github.com/crossbario/autobahn-python/pull/955/files#diff-47756a33cbfe991b9b5b45233a7a3b1aR81

fixed via https://github.com/crossbario/autobahn-python/pull/955

[dol-sen]

Thank you for the prompt fix :) unfortunately I have other fire suppression to do today at work, so likely won't get a change to apply the patch and finish the release in Gentoo today... possibly tomorrow, then on to crossbar and buildbot releases...

[oberstet]

np, thx for letting us know!