Cannot run with linux

[lwshang]

I'm trying to install jupyterq. But it can't run. In jupyter notebook, the kernel dies immediately after I open a new Q notebook and tries to restart but fail.

Then I try the jupyter console with:

$ jupyter console --kernel=qpk

There is no response. So I suspend the process with ctrl + C. It shows the error message as below:

Traceback (most recent call last):
  File "/home/lshang/.local/bin/jupyter-console", line 11, in <module>
    sys.exit(main())
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_core/application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/lshang/.local/lib/python3.5/site-packages/traitlets/config/application.py", line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-116>", line 2, in initialize
  File "/home/lshang/.local/lib/python3.5/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/app.py", line 141, in initialize
    self.init_shell()
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/app.py", line 114, in init_shell
    client=self.kernel_client,
  File "/home/lshang/.local/lib/python3.5/site-packages/traitlets/config/configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/ptshell.py", line 272, in __init__
    self.init_kernel_info()
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/ptshell.py", line 323, in init_kernel_info
    reply = self.client.get_shell_msg(timeout=1)
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_client/client.py", line 77, in get_shell_msg
    return self.shell_channel.get_msg(*args, **kwargs)
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_client/blocking/channels.py", line 50, in get_msg
    ready = self.socket.poll(timeout)
  File "/home/lshang/.local/lib/python3.5/site-packages/zmq/sugar/socket.py", line 695, in poll
    evts = dict(p.poll(timeout))
  File "/home/lshang/.local/lib/python3.5/site-packages/zmq/sugar/poll.py", line 99, in poll
    return zmq_poll(self.sockets, timeout=timeout)
  File "zmq/backend/cython/_poll.pyx", line 143, in zmq.backend.cython._poll.zmq_poll
  File "zmq/backend/cython/_poll.pyx", line 123, in zmq.backend.cython._poll.zmq_poll
  File "zmq/backend/cython/checkrc.pxd", line 12, in zmq.backend.cython.checkrc._check_rc
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/app.py", line 123, in handle_sigint
    if self.shell._executing:
AttributeError: 'ZMQTerminalIPythonApp' object has no attribute 'shell'

It seems like an issue with jupyter. But it works pretty well with Python or R kernel. So I believe it is more likely to be an issue here.

[jhanna-kx]

Hello, Could you confirm the kdb+ version you are using (.z.k and .z.K) and then can you run the jupyter console with the debug flag

 $ jupyter console --kernel=qpk --debug

And post any output, also please check you have the latest version of embedPy https://github.com/KxSystems/embedPy/releases/latest and jupyterq https://github.com/KxSystems/jupyterq/releases/latest

Thanks

[lwshang]

I'm using the 64-bit on-demand version of KDB.

q).z.k
2018.02.26
q).z.K
3.5

I checked the debug mode. The error messages are included below:

[ZMQTerminalIPythonApp] Searching ['/usr/local/bin', '/home/lshang/.jupyter', '/usr/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[ZMQTerminalIPythonApp] Looking for jupyter_config in /etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /usr/local/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /usr/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /home/lshang/.jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /usr/local/bin
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /usr/local/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /usr/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /home/lshang/.jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /usr/local/bin
[ZMQTerminalIPythonApp] Connection File not found: /run/user/1001/jupyter/kernel-17122.json
[ZMQTerminalIPythonApp] Starting kernel: ['q', 'jupyterq_kernel.q', '-cds', '/run/user/1001/jupyter/kernel-17122.json']
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:39923
[ZMQTerminalIPythonApp] connecting shell channel to tcp://127.0.0.1:58732
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:58732
[ZMQTerminalIPythonApp] connecting iopub channel to tcp://127.0.0.1:35776
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:35776
[ZMQTerminalIPythonApp] connecting stdin channel to tcp://127.0.0.1:34991
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:34991
[ZMQTerminalIPythonApp] connecting heartbeat channel to tcp://127.0.0.1:40576
Traceback (most recent call last):
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/ptshell.py", line 323, in init_kernel_info
    reply = self.client.get_shell_msg(timeout=1)
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_client/client.py", line 77, in get_shell_msg
    return self.shell_channel.get_msg(*args, **kwargs)
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_client/blocking/channels.py", line 57, in get_msg
    raise Empty
queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/lshang/.local/bin/jupyter-console", line 11, in <module>
    sys.exit(main())
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_core/application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/lshang/.local/lib/python3.5/site-packages/traitlets/config/application.py", line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-116>", line 2, in initialize
  File "/home/lshang/.local/lib/python3.5/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/app.py", line 141, in initialize
    self.init_shell()
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/app.py", line 114, in init_shell
    client=self.kernel_client,
  File "/home/lshang/.local/lib/python3.5/site-packages/traitlets/config/configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/ptshell.py", line 272, in __init__
    self.init_kernel_info()
  File "/home/lshang/.local/lib/python3.5/site-packages/jupyter_console/ptshell.py", line 326, in init_kernel_info
    raise RuntimeError("Kernel didn't respond to kernel_info_request")
RuntimeError: Kernel didn't respond to kernel_info_request

[jhanna-kx]

It appears your kernel is not starting at all, haven't been able to replicate the issue but the latest release has some additional startup logging which should help find where the failure is, could you download it from here

https://github.com/KxSystems/embedPy/releases/tag/1.1.2

And then run JUPYTERQ_DEBUG=1 jupyter-console --kernel=qpk

The start of the expected output is

jupyterq_kernel: loading embedPy
jupyterq_kernel: loading pyzmq
jupyterq_kernel: zeromq socket setup
jupyterq_kernel: check imports
jupyterq_kernel: start server
jupyterq_kernel: completed loading
`header`pheader`metadata`content!(`version`date`session`username`msg_type`msg..
`header`pheader`metadata`content!(`version`date`session`username`msg_type`msg..
`header`pheader`metadata`content!(`version`date`session`username`msg_type`msg..
`header`pheader`metadata`content!(`version`date`session`username`msg_type`msg..
`header`pheader`metadata`content!(`version`date`session`username`msg_type`msg..
`header`pheader`metadata`content!(`version`date`session`username`msg_type`msg..
`header`pheader`metadata`content!(`version`date`session`username`msg_type`msg..
Unrecognized message type history_request on channel sh

Could you post your output? Apologies for the trouble Thanks

[lwshang]

I used system python 3.5 before. After switch to anaconda python 3.6.4, the kernel works. After start the jupyter console, there is a warning message as below. Is it normal?

Unrecognized message type history_request on channel sh
/home/lshang/anaconda3/lib/python3.6/site-packages/jupyter_console/ptshell.py:95: UserWarning: No lexer found for language 'q'. Treating as plain text.
  warn("No lexer found for language %r. Treating as plain text." % name)

[jhanna-kx]

Yes that warning is normal, there is no pygments lexer for q as far as I'm aware

[lwshang]

Thanks!

During my setup procedure, I once got an error message from jupyterq which complaining that it can't find "q" in my PATH even though I set alias for q as the typical way. Therefore, I created an executable script in one of my PATH directory which execute q. Then I overcome that error.

Is it an necessary step to setup jupyterq? If the answer is yes, I suggest that you add it to the README file so that other users may save some time.

[jhanna-kx]

Hello, Yes q needs to be on the PATH during install and more importantly at runtime since jupyter runs q, an alias doesn't work for jupyter to be able to do this unless I'm missing something

Without q on the PATH.

$ alias q
alias q='$QHOME/l64/q '
$ echo '`hello' |q 
`hello
$ jupyter-console --kernel=qpk
[ZMQTerminalIPythonApp] ERROR | Failed to run command:
['q', 'jupyterq_kernel.q', '-cds', '/run/user/1000/jupyter/kernel-24371.json']
... 
more error messages
FileNotFoundError: [Errno 2] No such file or directory: 'q': 'q'

We'll add a note to the docs for this, typically I'd put $QHOME/l64 (or m64) on the PATH and then alias q with rlwrap if you're using that.