undefined symbol error installing pyq 4.1 on ubuntu

[alexzhang2010]

I got the following errors while installing pyq on ubuntu (Linux ubuntu.fmr.com 4.4.0-96-generic #119-Ubuntu SMP Tue Sep 12 14:59:54 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux). I am upgrading from pyq 4.0.2. I can confirm 4.0.2 does not have this issue.

Thanks, Ray

(env_py3) root@ubuntu:/opt# pyq
Traceback (most recent call last):
  File "/opt/anaconda3/envs/env_py3/lib/python3.6/site-packages/pyq/__init__.py", line 22, in <module>
    from ._k import K as _K, error as kerr, Q_VERSION, Q_DATE, Q_OS
ImportError: /opt/anaconda3/envs/env_py3/lib/python3.6/site-packages/pyq/_k.cpython-36m-x86_64-linux-gnu.so: undefined symbol: clr

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/anaconda3/envs/env_py3/lib/python3.6/site-packages/pyq/__init__.py", line 31, in <module>
    raise ImportError(message)
ImportError: Importing pyq from stock python is not supported. Use pyq executable.
'2017.09.21T12:42:31.611 python

[abalkin]

@alexzhang2010, from the error message, it looks like you have compiled pyq with kdb+ version < 3.5, but run it on kdb+ 3.5. Please follow the published instructions and install both kdb+ and pyq in a virtual environment. It is important that you run pyq with the same version of kdb+ that was used to build it.

[alexzhang2010]

Thanks for the quick response. Actually, my kdb version is 3.4

$q KDB+ 3.4 2017.01.17 Copyright (C) 1993-2017 Kx Systems

and I installed pyq in a conda environment.

pip install -i https://pyq.enlnt.com --no-binary pyq pyq

[abalkin]

OK, to be sure of the versions, please post pyq --versions output.

Anaconda has its own quirks and we normally don't recommend using it, but on Ubuntu using system supplied Python with pyq is probably even harder. In any case, If you follow our Ubuntu installation instructions you should get a setup similar to what we test internally.

This said, you should not see undefined symbol: clr if you use kdb+ 3.4. What do you see if you enter

nm $VIRTUAL_ENV/q/l64/q | grep clr

Also, please try starting pyq by running the following command

QHOME=$VIRTUAL_ENV/q $VIRTUAL_ENV/q/l64/q python.q

This should give you the python prompt. Please enter

>>> import sys
>>> sys.executable

and let me know what you see.

[abalkin]

@alexzhang2010, thank you for including the date of your the kdb+ release that you are using, 2017.01.17. It turns out, we are testing kdb+ 3.4 using the 2016.11.18 edition. I have just checked and indeed Kx made significant changes between 2016.11.18 and 2017.01.17 to the q binary:

$ ls -l 3.4/*/l64/q
-rwxr-xr-x 1 a a 597195 Nov 18  2016 3.4/20161118/l64/q
-rwxr-xr-x 1 a a 535597 Jan 17  2017 3.4/20170117/l64/q

(notice how the binary became 10% smaller!)

We should be able to issue a patch to work around these changes, but meanwhile I suggest that you either upgrade to kdb+ 3.5 or downgrade to 3.4 2016.11.18.

[abalkin]

Attached patch eliminates all uses of clr and was tested with KDB+ 3.4 (2017.01.17) l64:

$ pyq --versions
PyQ 4.2.0.dev14+c5fb0b4
KDB+ 3.4 (2017.01.17) l64
Python 2.7.13 (default, Jan  4 2017, 08:35:45)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]

0001-BUG-Fixes-956-Do-not-use-clr.patch.txt

[abalkin]

Fixed in the internal issue Nº 956.