[python] library is installed in a wrong directory with --user flag

[tunz]

version: 0.9.1

If I install keystone-engine with --user flag without virtualenv,

$ pip install --user keystone-engine

libkeystone.so is installed in a wrong directory, thus loading the keystone-engine throws an exception:

ImportError: ERROR: fail to load the dynamic library.

the libkeystone.so is installed in /home/username/.local/lib/python2.7/site-packages/usr/lib/python2.7/dist-packages/keystone/libkeystone.so, but it should be located in /home/username/.local/lib/python2.7/site-packages/keystone/libkeystone.so

it seems that capstone has the same issue.

[aquynh]

Did you try this on latest Capstone code? If you did, can you open a new issue on Capstone repo?

[tunz]

it's broken in pip-installed version (3.0.4) of capstone, but seems it's fixed in the master branch of capstone.

[aquynh]

we can fix this by porting the the Python binding of Unicorn to Keystone (as they are very similar). the only related Keystone file is https://github.com/keystone-engine/keystone/blob/master/bindings/python/setup.py, and the counterpart is https://github.com/unicorn-engine/unicorn/blob/master/bindings/python/setup.py

if you can fix this, please send pull req

[williballenthin]

with #301 merged, i think this issue is resolved. see the below terminal session building keystone on linux/python3:

before patch:

localhost:python (master*) $ git checkout d19579217c0ac71e2d63deb389f29fb23e5a9118                                                         
Note: checking out 'd19579217c0ac71e2d63deb389f29fb23e5a9118'.                                                                           

You are in 'detached HEAD' state. You can look around, make experimental                                                                      
changes and commit them, and you can discard any commits you make in this                                                                      
state without impacting any branches by performing another checkout.                                                                         

If you want to create a new branch to retain commits you create, you may                                                                       
do so (now or later) by using -b with the checkout command again. Example:                                                                   

  git checkout -b <new-branch-name>                                                                                                     

HEAD is now at d195792... clean travis and add linux (#315)                                                                                
localhost:python (d195792*) $ /tmp/ks/pre-env/bin/python setup.py sdist --format=zip                                                    
running sdist                                                                                                                           
removing 'src/' (and everything under it)                                                                                               
creating src                                                                                                                            
creating src/llvm                                                                                                                       
creating src/llvm/utils                                                                                                                 
creating src/llvm/utils/llvm-build                                                                                                      
creating src/llvm/utils/llvm-build/llvmbuild                                                                                          
copying ../../llvm/utils/llvm-build/llvmbuild/configutil.pyc -> src/llvm/utils/llvm-build/llvmbuild                                     
copying ../../llvm/utils/llvm-build/llvmbuild/util.pyc -> src/llvm/utils/llvm-build/llvmbuild                                           
copying ../../llvm/utils/llvm-build/llvmbuild/componentinfo.pyc -> src/llvm/utils/llvm-build/llvmbuild                                            
copying ../../llvm/utils/llvm-build/llvmbuild/main.pyc -> src/llvm/utils/llvm-build/llvmbuild                                                     
copying ../../llvm/utils/llvm-build/llvmbuild/__init__.pyc -> src/llvm/utils/llvm-build/llvmbuild                                        
copying ../../llvm/utils/llvm-build/llvmbuild/util.py -> src/llvm/utils/llvm-build/llvmbuild                                                  
copying ../../llvm/utils/llvm-build/llvmbuild/main.py -> src/llvm/utils/llvm-build/llvmbuild                                            
copying ../../llvm/utils/llvm-build/llvmbuild/configutil.py -> src/llvm/utils/llvm-build/llvm
[...]

localhost:dist (d195792*) $ /tmp/ks/pre-env/bin/pip install ./keystone-engine-0.9.1-3.zip                                                                                                                                                                                       
Processing ./keystone-engine-0.9.1-3.zip                            
Building wheels for collected packages: keystone-engine             
  Running setup.py bdist_wheel for keystone-engine ... done         
  Stored in directory: /home/user/.cache/pip/wheels/99/d4/10/207a426f6fcfca9dfada77be492a7920c67f98909c3e490f01                         
Successfully built keystone-engine                                  
Installing collected packages: keystone-engine                      
Successfully installed keystone-engine-0.9.1.post3                  
/tmp/ks/pre-env/bin/pip install ./keystone-engine-0.9.1-3.zip  280.23s user 19.77s system 527% cpu 56.853 total 
localhost:dist (d195792*) $ /tmp/ks/pre-env/bin/python                                
Python 3.6.2 (default, Oct  2 2017, 16:51:32) 
[GCC 7.2.1 20170915 (Red Hat 7.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import keystone
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/ks/pre-env/lib/python3.6/site-packages/keystone/__init__.py", line 4, in <module>
    from .keystone import Ks, ks_version, ks_arch_supported, version_bind, debug, KsError, __version__
  File "/tmp/ks/pre-env/lib/python3.6/site-packages/keystone/keystone.py", line 74, in <module>
    raise ImportError("ERROR: fail to load the dynamic library.")
ImportError: ERROR: fail to load the dynamic library.
>>> 

after patch:

localhost:python (d195792) $ git checkout master
localhost:python (master) $ /tmp/ks/post-env/bin/python setup.py sdist --format=zip                                                
running sdist                                                                                                                   
removing 'src/' (and everything under it)                                                                                             
creating src                                                                                                                         
creating src/llvm                                                                                                                       
creating src/llvm/utils                                                                                                                 
creating src/llvm/utils/llvm-build                                                                                                                                                                                                                                              
creating src/llvm/utils/llvm-build/llvmbuild                                                                                          
copying ../../llvm/utils/llvm-build/llvmbuild/configutil.pyc -> src/llvm/utils/llvm-build/llvmbuild                                              
copying ../../llvm/utils/llvm-build/llvmbuild/util.pyc -> src/llvm/utils/llvm-build/llvmbuild  
[...]

localhost:python (master*) $ cd dist 
localhost:dist (master*) $ ls                                       
keystone-engine-0.9.1-3.zip                                         
localhost:dist (master*) $ /tmp/ks/post-env/bin/pip install ./keystone-engine-0.9.1-3.zip                                               
Processing ./keystone-engine-0.9.1-3.zip                            
Building wheels for collected packages: keystone-engine             
  Running setup.py bdist_wheel for keystone-engine ... done         
  Stored in directory: /home/user/.cache/pip/wheels/99/d4/10/207a426f6fcfca9dfada77be492a7920c67f98909c3e490f01                         
Successfully built keystone-engine                                  
Installing collected packages: keystone-engine                      
Successfully installed keystone-engine-0.9.1.post3                  
/tmp/ks/post-env/bin/pip install ./keystone-engine-0.9.1-3.zip  283.91s user 20.02s system 505% cpu 1:00.09 total                       
localhost:dist (master*) $ /tmp/ks/post-env/bin/python                                   
Python 3.6.2 (default, Oct  2 2017, 16:51:32) 
[GCC 7.2.1 20170915 (Red Hat 7.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import keystone
>>> 

[edevil]

Had this problem today on Ubuntu 18.04.2 LTS.