Open asyne opened 9 years ago
+
Ok. So I have built aerospike-client-python using CPython: python setup.py build --force . And then tried installing it to PyPy's site-packages with pypy setup.py install --force. Is it correct?
The result, by the way, is the same, but error have changed in some places:
Traceback (most recent call last):
File "main.py", line 2, in <module>
import aerospike
File "/Users/food/Applications/pypy-2.6.1/site-packages/aerospike-1.0.53-py2.7-macosx-10.11-x86_64.egg/aerospike.py", line 7, in <module>
__bootstrap__()
File "/Users/food/Applications/pypy-2.6.1/site-packages/aerospike-1.0.53-py2.7-macosx-10.11-x86_64.egg/aerospike.py", line 6, in __bootstrap__
imp.load_dynamic(__name__,__file__)
ImportError: unable to load extension module '/Users/food/Applications/pypy-2.6.1/site-packages/aerospike-1.0.53-py2.7-macosx-10.11-x86_64.egg/aerospike.pypy-26.so': dlopen(/Users/food/Applications/pypy-2.6.1/site-packages/aerospike-1.0.53-py2.7-macosx-10.11-x86_64.egg/aerospike.pypy-26.so, 6): Symbol not found: _PyByteArray_AsString
Referenced from: /Users/food/Applications/pypy-2.6.1/site-packages/aerospike-1.0.53-py2.7-macosx-10.11-x86_64.egg/aerospike.pypy-26.so
Expected in: flat namespace
in /Users/food/Applications/pypy-2.6.1/site-packages/aerospike-1.0.53-py2.7-macosx-10.11-x86_64.egg/aerospike.pypy-26.so
BTW, I also tried to build and install aerospike client using CPython and then move package from CPython to PyPy site-packages. Still have same problem.
Well, it's a C extension for CPython, meaning it uses its C API. PyPy runs pure Python code, so it cannot use our module. I'm looking into ways of getting our module to work with PyPy.
I see. Maybe there is some way for me to call CPython package directly from PyPy code? Something like running through cmd, but native, e.g. execfile()?
The main problem that PyPy is missing some types from CPython like PyBool, PyByteArray, etc. That is kinda obvious. Temporary solving for this is to call CPython from PyPy using PyExt or PyMetabiosis (which was suggested to use on PyPy's IRC channel) extension. This is how it works in PyPy with PyMetabiosis:
import pymetabiosis
import aspike_config
aerospike = pymetabiosis.module.import_module("aerospike")
client = aerospike.client(aspike_config.get_config())
client.connect()
...
If you can get such a project going I'd be happy to promote it. I was looking into CFFI as a more generic solution, since it works with more runtimes than just PyPy.
A few weeks a go I hacked the driver to avoid using the ByteArray functions/types, very hacky but it worked well enough for our application (except for the callback based parts like query and scan).
Final performance was bad though (as expected from the CPython emulation layer). I'm almost sure it had some memory leaks too. You can check the code here https://github.com/arthurprs/aerospike-client-python/tree/pypy
Finally, unless PYPY makes the rest of you application fly I recommend avoiding it unless you can find an wrapper that adds less overhead (pymetabiosis is probably even slower).
System info: ProductName: Mac OS X ProductVersion: 10.11 BuildVersion: 15A284
LLVM info: Apple LLVM version 7.0.0 (clang-700.0.72) Target: x86_64-apple-darwin15.0.0 Thread model: posix
PyPy & pip info: Python 2.7.10 (f3ad1e1e1d62, Aug 28 2015, 09:36:42) [PyPy 2.6.1 with GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin