oberstet
commented
5 years ago fwiw, wsaccel does not even build on pypy (currently):
(pypy3_1) oberstet@matterhorn:~/scm/crossbario/crossbar-examples/benchmark/websocket$ pip install wsaccel
Collecting wsaccel
Cache entry deserialization failed, entry ignored
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/52/46/9ef0744b434ac723ae3aeddcb92ab29af14f8912f53f47590b5a0db0b9d6/wsaccel-0.6.2.tar.gz
Building wheels for collected packages: wsaccel
Running setup.py bdist_wheel for wsaccel ... error
Complete output from command /home/oberstet/python/pypy3_1/bin/pypy3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-hh2fkijq/wsaccel/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpue9cfm_6pip-wheel- --python-tag pp370:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.5
creating build/lib.linux-x86_64-3.5/wsaccel
copying wsaccel/__init__.py -> build/lib.linux-x86_64-3.5/wsaccel
running build_ext
building 'wsaccel.utf8validator' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/wsaccel
gcc -pthread -DNDEBUG -O2 -fPIC -I/home/oberstet/python/pypy3_1/include -I/home/oberstet/python/pypy3.5-v7.0.0-linux64/include -c wsaccel/utf8validator.c -o build/temp.linux-x86_64-3.5/wsaccel/utf8validator.o
wsaccel/utf8validator.c:235:0: warning: "PyMethod_New" redefined
#define PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : PyInstanceMethod_New(func))
In file included from /home/oberstet/python/pypy3.5-v7.0.0-linux64/include/Python.h:148:0,
from wsaccel/utf8validator.c:16:
/home/oberstet/python/pypy3.5-v7.0.0-linux64/include/pypy_decl.h:487:0: note: this is the location of the previous definition
#define PyMethod_New PyPyMethod_New
wsaccel/utf8validator.c: In function ‘__pyx_pf_7wsaccel_13utf8validator_13Utf8Validator_6validate’:
wsaccel/utf8validator.c:983:49: warning: passing argument 2 of ‘PyPyObject_AsReadBuffer’ from incompatible pointer type [-Wincompatible-pointer-types]
__pyx_t_1 = PyObject_AsReadBuffer(__pyx_v_ba, ((void **)(&__pyx_v_buf)), (&__pyx_v_buf_len)); if (unlikely(__pyx_t_1 == -1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 104; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
^
In file included from /home/oberstet/python/pypy3.5-v7.0.0-linux64/include/Python.h:76:0,
from wsaccel/utf8validator.c:16:
/home/oberstet/python/pypy3.5-v7.0.0-linux64/include/pypy_macros.h:66:31: note: expected ‘const void **’ but argument is of type ‘void **’
#define PyObject_AsReadBuffer PyPyObject_AsReadBuffer
^
/home/oberstet/python/pypy3.5-v7.0.0-linux64/include/object.h:333:17: note: in expansion of macro ‘PyObject_AsReadBuffer’
PyAPI_FUNC(int) PyObject_AsReadBuffer(PyObject *, const void **, Py_ssize_t *);
^~~~~~~~~~~~~~~~~~~~~
gcc -pthread -shared build/temp.linux-x86_64-3.5/wsaccel/utf8validator.o -o build/lib.linux-x86_64-3.5/wsaccel/utf8validator.pypy3-70-x86_64-linux-gnu.so
building 'wsaccel.xormask' extension
gcc -pthread -DNDEBUG -O2 -fPIC -I/home/oberstet/python/pypy3_1/include -I/home/oberstet/python/pypy3.5-v7.0.0-linux64/include -c wsaccel/xormask.c -o build/temp.linux-x86_64-3.5/wsaccel/xormask.o
wsaccel/xormask.c:235:0: warning: "PyMethod_New" redefined
#define PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : PyInstanceMethod_New(func))
In file included from /home/oberstet/python/pypy3.5-v7.0.0-linux64/include/Python.h:148:0,
from wsaccel/xormask.c:16:
/home/oberstet/python/pypy3.5-v7.0.0-linux64/include/pypy_decl.h:487:0: note: this is the location of the previous definition
#define PyMethod_New PyPyMethod_New
wsaccel/xormask.c: In function ‘__Pyx_Raise’:
wsaccel/xormask.c:2165:34: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘curexc_traceback’
PyObject* tmp_tb = tstate->curexc_traceback;
^~
wsaccel/xormask.c:2168:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘curexc_traceback’
tstate->curexc_traceback = tb;
^~
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for wsaccel
Running setup.py clean for wsaccel
Failed to build wsaccel
Installing collected packages: wsaccel
Running setup.py install for wsaccel ... error
Complete output from command /home/oberstet/python/pypy3_1/bin/pypy3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-hh2fkijq/wsaccel/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-tihzbgfl-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/oberstet/python/pypy3_1/include/site/python3.5/wsaccel:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.5
creating build/lib.linux-x86_64-3.5/wsaccel
copying wsaccel/__init__.py -> build/lib.linux-x86_64-3.5/wsaccel
running build_ext
building 'wsaccel.utf8validator' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/wsaccel
gcc -pthread -DNDEBUG -O2 -fPIC -I/home/oberstet/python/pypy3_1/include -I/home/oberstet/python/pypy3.5-v7.0.0-linux64/include -c wsaccel/utf8validator.c -o build/temp.linux-x86_64-3.5/wsaccel/utf8validator.o
wsaccel/utf8validator.c:235:0: warning: "PyMethod_New" redefined
#define PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : PyInstanceMethod_New(func))
In file included from /home/oberstet/python/pypy3.5-v7.0.0-linux64/include/Python.h:148:0,
from wsaccel/utf8validator.c:16:
/home/oberstet/python/pypy3.5-v7.0.0-linux64/include/pypy_decl.h:487:0: note: this is the location of the previous definition
#define PyMethod_New PyPyMethod_New
wsaccel/utf8validator.c: In function ‘__pyx_pf_7wsaccel_13utf8validator_13Utf8Validator_6validate’:
wsaccel/utf8validator.c:983:49: warning: passing argument 2 of ‘PyPyObject_AsReadBuffer’ from incompatible pointer type [-Wincompatible-pointer-types]
__pyx_t_1 = PyObject_AsReadBuffer(__pyx_v_ba, ((void **)(&__pyx_v_buf)), (&__pyx_v_buf_len)); if (unlikely(__pyx_t_1 == -1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 104; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
^
In file included from /home/oberstet/python/pypy3.5-v7.0.0-linux64/include/Python.h:76:0,
from wsaccel/utf8validator.c:16:
/home/oberstet/python/pypy3.5-v7.0.0-linux64/include/pypy_macros.h:66:31: note: expected ‘const void **’ but argument is of type ‘void **’
#define PyObject_AsReadBuffer PyPyObject_AsReadBuffer
^
/home/oberstet/python/pypy3.5-v7.0.0-linux64/include/object.h:333:17: note: in expansion of macro ‘PyObject_AsReadBuffer’
PyAPI_FUNC(int) PyObject_AsReadBuffer(PyObject *, const void **, Py_ssize_t *);
^~~~~~~~~~~~~~~~~~~~~
gcc -pthread -shared build/temp.linux-x86_64-3.5/wsaccel/utf8validator.o -o build/lib.linux-x86_64-3.5/wsaccel/utf8validator.pypy3-70-x86_64-linux-gnu.so
building 'wsaccel.xormask' extension
gcc -pthread -DNDEBUG -O2 -fPIC -I/home/oberstet/python/pypy3_1/include -I/home/oberstet/python/pypy3.5-v7.0.0-linux64/include -c wsaccel/xormask.c -o build/temp.linux-x86_64-3.5/wsaccel/xormask.o
wsaccel/xormask.c:235:0: warning: "PyMethod_New" redefined
#define PyMethod_New(func, self, klass) ((self) ? PyMethod_New(func, self) : PyInstanceMethod_New(func))
In file included from /home/oberstet/python/pypy3.5-v7.0.0-linux64/include/Python.h:148:0,
from wsaccel/xormask.c:16:
/home/oberstet/python/pypy3.5-v7.0.0-linux64/include/pypy_decl.h:487:0: note: this is the location of the previous definition
#define PyMethod_New PyPyMethod_New
wsaccel/xormask.c: In function ‘__Pyx_Raise’:
wsaccel/xormask.c:2165:34: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘curexc_traceback’
PyObject* tmp_tb = tstate->curexc_traceback;
^~
wsaccel/xormask.c:2168:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘curexc_traceback’
tstate->curexc_traceback = tb;
^~
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/home/oberstet/python/pypy3_1/bin/pypy3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-hh2fkijq/wsaccel/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-tihzbgfl-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/oberstet/python/pypy3_1/include/site/python3.5/wsaccel" failed with error code 1 in /tmp/pip-build-hh2fkijq/wsaccel/
You are using pip version 9.0.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(pypy3_1) oberstet@matterhorn:~/scm/crossbario/crossbar-examples/benchmark/websocket$
The PR https://github.com/crossbario/autobahn-python/pull/967 has merged the Autobahn Native Vector Extensions (NXP), which include an accelerated UTF8 validator.
We should make use of this when available in the Autobahn WebSocket layer. The native SIMD code has significant performance speedup versus CPython/PyPy/Cython/..