search

KxSystems / pyq

PyQ — Python for kdb+
http://code.kx.com/q/interfaces
Apache License 2.0
191 stars 49 forks source link

installing pyq on redhat 2.7 with anaconda 5.0 #32

Closed alexzhang2010 closed 6 years ago

alexzhang2010 commented 6 years ago

I tried to install pyq with python 3.6 under anaconda 5.0. I got the following error:

Processing /apps/download/pyq-master
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/pyq.egg-info
    writing pip-egg-info/pyq.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/pyq.egg-info/dependency_links.txt
    writing requirements to pip-egg-info/pyq.egg-info/requires.txt
    writing top-level names to pip-egg-info/pyq.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/pyq.egg-info/SOURCES.txt'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-29sp_qa8-build/setup.py", line 658, in <module>
        run_setup(METADATA)
      File "/tmp/pip-29sp_qa8-build/setup.py", line 654, in run_setup
        setup(**keywords)
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 280, in run
        self.find_sources()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 295, in find_sources
        mm.run()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 526, in run
        self.add_defaults()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 562, in add_defaults
        sdist.add_defaults(self)
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults
        self._add_defaults_ext()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
        build_ext = self.get_finalized_command('build_ext')
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/cmd.py", line 299, in get_finalized_command
        cmd_obj.ensure_finalized()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/cmd.py", line 107, in ensure_finalized
        self.finalize_options()
      File "/tmp/pip-29sp_qa8-build/setup.py", line 449, in finalize_options
        ('define', 'define'))
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/cmd.py", line 287, in set_undefined_options
        src_cmd_obj.ensure_finalized()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/cmd.py", line 107, in ensure_finalized
        self.finalize_options()
      File "/tmp/pip-29sp_qa8-build/setup.py", line 384, in finalize_options
        ('q_version', 'q_version'))
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/cmd.py", line 287, in set_undefined_options
        src_cmd_obj.ensure_finalized()
      File "/apps/anaconda3/envs/env_py3/lib/python3.6/distutils/cmd.py", line 107, in ensure_finalized
        self.finalize_options()
      File "/tmp/pip-29sp_qa8-build/setup.py", line 284, in finalize_options
        self.python_dll = get_python_dll(sys.executable)
      File "/tmp/pip-29sp_qa8-build/setup.py", line 242, in get_python_dll
        raise RuntimeError('no python dll')
    RuntimeError: no python dll

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-29sp_qa8-build/

I took a look at setup.py, looks like pyq installer is looking for python shared library libpython3.6m.so by using ldd But in this case, the share lib has been statically compiled into python3.6 executable. How do I get around this issue?

/apps/anaconda3/lib$ ldd /apps/anaconda3/bin/python
    linux-vdso.so.1 =>  (0x00007ffd2fd37000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb7daf6d000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fb7dabab000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007fb7da9a7000)
    libutil.so.1 => /lib64/libutil.so.1 (0x00007fb7da7a4000)
    librt.so.1 => /lib64/librt.so.1 (0x00007fb7da59b000)
    libm.so.6 => /lib64/libm.so.6 (0x00007fb7da299000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fb7db6ea000)
sashkab commented 6 years ago

You might want to try code changes from #29.

abalkin commented 6 years ago

We are working on improving the situation for conda users. Meanwhile, the workaround is to add a setup.cfg file to the root of the source tree with the following contents:

[config]
python_dll=libpython3.6m.so

See Installing from source code for details on how to obtain PyQ source code.

stanleyxu2005 commented 6 years ago

I tried to install via source by having this setup.cfg. But I got some other error message

[root@MiWiFi-R2D-srv ~]# python
Python 3.6.3 |Anaconda, Inc.| (default, Oct 13 2017, 14:21:34) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyq
Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.6/site-packages/pyq/__init__.py", line 22, in <module>
    from ._k import K as _K, error as kerr, Q_VERSION, Q_DATE, Q_OS
ImportError: /root/miniconda3/lib/python3.6/site-packages/pyq/_k.cpython-36m-i386-linux-gnu.so: undefined symbol: dot

I'm using Miniconda3 (x86) on CentOS 6.9 (x64). kdb+ binary is 32 bit.

abalkin commented 6 years ago

This is odd. What version of kdb+ do you use? What do you see when you enter the following command?

nm $QHOME/l32/q | grep dot

Please note that we recently started using Miniconda in the Travis CI builds. You can follow the steps from the log or from .travis.yml.

stanleyxu2005 commented 6 years ago

@abalkin if I simply run nm $QHOME/l32/q, it returns 080633e0 t _sn.

Second I also see this message when I try to import pyq in Python. ImportError: Importing pyq from stock python is not supported. Use pyq executable. So if I run pyq, it will launch another Python shell, which is able to run import pyq.

abalkin commented 6 years ago

Starting with release, 4.1.4 use

conda install -c kx -c enlnt pyq

to install both kdb+ and PyQ.