crossbario / autobahn-testsuite

Autobahn WebSocket protocol testsuite
https://crossbar.io/autobahn/
Apache License 2.0
980 stars 84 forks source link

wstest is a program to be installed into a dedicated venv #55

Open JackDunaway opened 8 years ago

JackDunaway commented 8 years ago

Recent report here: http://stackoverflow.com/questions/34157314/autobahn-websocket-issue-while-running-with-twistd-using-tac-file

meejah commented 8 years ago

Thanks for the report! I think all that needs to happen is an Autobahn release containing dd1cff8f9480a125e510108482bfaf5b9eedc916

oberstet commented 8 years ago

AutobahnTestsuite is pinned https://github.com/crossbario/autobahn-testsuite/blob/master/autobahntestsuite/setup.py#L87

We need to pin txaio then also.

meejah commented 8 years ago

I think anyone doing a fresh install of autobahn (or anything requiring it) right now will get the newest txaio, which will not work.

But yes, limiting txaio to <= 2.1.0 should work for autoban-testsuite.

oberstet commented 8 years ago

I think anyone doing a fresh install ...

Ah, right. so essentially, we must do a release of Autobahn. The API still isn't done - well. What version should bump to?

Minor: why do I get the following locally, but Travis is green?

(python279_1)oberstet@thinkpad-t430s:~/scm/crossbario/autobahn-python$ make flake8
flake8 --ignore=E501,N801,N802,N803,N805,N806 autobahn
autobahn/wamp/test/test_protocol_peer.py:39:1: E402 module level import not at top of file
autobahn/wamp/test/test_protocol_peer.py:40:1: E402 module level import not at top of file
autobahn/wamp/test/test_protocol_peer.py:41:1: E402 module level import not at top of file
autobahn/wamp/test/test_protocol_peer.py:42:1: E402 module level import not at top of file
autobahn/wamp/test/test_protocol_peer.py:44:1: E402 module level import not at top of file
autobahn/asyncio/websocket.py:34:1: E402 module level import not at top of file
autobahn/asyncio/websocket.py:35:1: E402 module level import not at top of file
autobahn/asyncio/websocket.py:48:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:36:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:37:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:38:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:39:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:42:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:43:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:45:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:46:1: E402 module level import not at top of file
autobahn/twisted/websocket.py:48:1: E402 module level import not at top of file
make: *** [flake8] Fehler 1
JackDunaway commented 8 years ago

Ah, right. so essentially, we must do a release of Autobahn. The API still isn't done - well.

It appears this issue #55 is not the simple 1-liner fix in this repo (not another lib) I thought it might be -- that said, what's your advice on fresh installs of Autobahn TestSuite in the meantime? (one valid option is to just wait, if a release of Autobahn is reasonably close). Thanks!

oberstet commented 8 years ago

@wirebirdlabs https://pypi.python.org/pypi/autobahntestsuite/0.7.4 contains the fix. could you check if that works for you?

JackDunaway commented 8 years ago

Looks good! Much appreciated.

jchampio commented 8 years ago

pip apparently doesn't try to actually resolve dependencies -- it's first-come, first-serve -- so 0.7.4 does not fix this for me (since autobahn still declares txaio>=1.1.0).

meejah commented 8 years ago

@jchampio can you describe what you did? I did pip install autobahntestsuite in a fresh virtualenv and get the correct txaio version (2.1.0).

oberstet commented 8 years ago

I think we should pin all dep versions to exact versions, and document the recommended installation is via a dedicated virtualenv where the wstest command will live. The latter is the only user level interaction surface, and there is not public library level API. So wstest should be considered a blackbox "program", and deps are fully internal business. Maybe such an approach might allow us to decouple releases. But yes, we should also look at the actual incompat. at some point - otherwise the testsuite would be stuck on old Autobahn, and we then sooner or later run into the question of supporting a maintenance branch on Autobahn. Maybe the latter is unavoidable with the new LTS API. Well, what do you think?

jchampio commented 8 years ago

@meejah > pip install -r requirements.txt

with requirements.txt:

autobahn~=0.10.7
autobahntestsuite~=0.7.4

Switching the order "fixes" the problem, obviously, but more complicated projects than mine might not have the option. Honestly I'm not sure if there is a "correct" fix for you, since pip doesn't resolve...

jchampio commented 8 years ago

I think we should pin all dep versions to exact versions

My two cents, worth what you paid for it... I have dealt with the "pinned versions" culture with Ruby, and I think it makes things incredibly brittle in the long term. For small projects it's fine, but heaven help you if two of your dependencies each rely on a slightly different version of the same module.

oberstet commented 8 years ago

@jchampio that can't happen, since autobahntestsuite is supposed to be installed in it's own virtualenv

JackDunaway commented 8 years ago

Treating Autobahn TestSuite like an application (rather than a library) with "private" dependencies works fine for our application; it's sitting inside CI environments that are rebuild from scratch. From the environments' perspectives, the dependency graph is ... a node (where subnodes are an opaque, private decision).

(BTW, @jchampio, long time no see; fancy running into each other here. Hope things are well with you!)

oberstet commented 8 years ago

I've reopened and changed the issue title. Goal is to

jchampio commented 8 years ago

@jchampio that can't happen, since autobahntestsuite is supposed to be installed in it's own virtualenv

@oberstet I have installed it in its own virtualenv, alongside the other pip modules that I need to be able to run the remainder of my testsuite.

Or are you saying that I can only run TestSuite in pure isolation from everything else? That prevents me from piping TestSuite's output to another Python module, etc., since I can't have two venvs active at the same time. It also makes it very difficult for a third party to set up their machine the same way I have, with a quick pip install -r requirements.txt.

@wirebirdlabs Doing well, thanks! I did a double-take when I saw Wirebird in my notifications, hehe.

oberstet commented 7 years ago

Or are you saying that I can only run TestSuite in pure isolation from everything else?

yes

since I can't have two venvs active at the same time

no need to: simply start your stuff by directly referring the respective python

brainwane commented 4 years ago

Hi! please try the beta of the new resolver in pip 20.2 and see whether that helps?

brainwane commented 3 years ago

pip 20.3 has the new dependency resolver on by default; please see the documentation on how to test and migrate in case it addresses this problem.