PyQ not picking up CPUS env variable when running as a Jupyter kernel

[jljs]

Questions

• [x] Which operating system are you using (if Linux, please provide flavour of it, i.e RedHat, CentOS or Ubuntu), is it 32-bit, or 64-bit? Red Hat Enterprise Linux Server release 6.9 (Santiago)

• [x] Which version of PyQ are you running? You can use pyq --versions or pip list | grep pyq. PyQ 4.1.3

• [x] Which version of kdb+ are you using, is it 32-bit or 64-bit? KDB+ 3.4 (2016.10.10) l64

• [x] If on 64-bit, is your QLIC set? Please provide output env | grep QLIC on linux/macOS, or set|grep QLIC on Windows. QLIC=/opt/kx/3.4

• [x] Did you use virtual environment to install PyQ? If not, why? Yes with Anaconda

• [x] Where is your QHOME? Please provide output env | grep QHOME on linux/macOS, or set|grep QHOME on Windows. QHOME=/opt/kx/3.4

• [x] Do you use Conda? If so, what version? Yes. conda 3.14.1

Steps to reproduce the issue

1) install pyq 3 as a kernel in Jupyter 2) set environment variable CPUS=3 in jupyter kernel.json (i.e. /home/dev/.local/share/jupyter/kernels/pyq_3/kernel.json) See http://jupyter-client.readthedocs.io/en/stable/kernels.html for more detail

This is what I have set

{
    "argv": [
        "/opt/kx/3.4/l64/q",
        "pyq-kernel.p",
        "1024",
        "65535",
        "{connection_file}"
    ],
    "display_name": "PyQ-3",
    "env": {
        "CPUS": "3",
        "QHOME": "/opt/kx/3.4",
        "QLIC": "/opt/kx/3.4"
    },
    "language": "python"
}

Expected result

Confirm that the variable has been picked up by starting a new notebook in Jupyter with PyQ 3 kernel. The new notebook should start successfully with kernel ready

Actual result

kernel error with the following error message from Jupyter

[I 17:25:41.354 NotebookApp] Creating new notebook in
[I 17:25:41.736 NotebookApp] Kernel started: 5c3c1f5f-2124-4507-b1bb-044e9e0af35f
'2018.05.08T17:25:41.745 cores
[I 17:25:44.737 NotebookApp] KernelRestarter: restarting kernel (1/5)
'2018.05.08T17:25:44.752 cores
[I 17:25:47.747 NotebookApp] KernelRestarter: restarting kernel (2/5)
'2018.05.08T17:25:47.761 cores
[I 17:25:50.758 NotebookApp] KernelRestarter: restarting kernel (3/5)
'2018.05.08T17:25:50.773 cores
[W 17:25:51.772 NotebookApp] Timeout waiting for kernel_info reply from 5c3c1f5f-2124-4507-b1bb-044e9e0af35f

Workaround

no workaround If you know workaround, please provide it here.

[abalkin]

The CPUS environment variable is only used by the pyq executable and the jupyter kernel calls q directly. In order to set the CPU affinity for the kernel process, you will have to replace "/opt/kx/3.4/l64/q" with a script that calls taskset.

[jljs]

Hi Abalkin,

Thank you for your reply. How would this change my installation? Could you please provide an example and a bit more detail on the installation instruction? Really appreciate this.

Thanks

Jeffrey

[abalkin]

Please try to change "argv" in your kernel.json file to

    "argv": [
        "taskset", "-c", "0",    
        "/opt/kx/3.4/l64/q",
        "pyq-kernel.p",
        "1024",
        "65535",
        "{connection_file}"
    ],

You can change "0" above to the desired list of processors.