search

KxSystems / jupyterq

Jupyter kernel for kdb+
https://code.kx.com/q/interfaces
Apache License 2.0
105 stars 46 forks source link

'cores error when starting kernel #10

Closed danielkrizian closed 6 years ago

danielkrizian commented 6 years ago

Hi, I am getting 'cores error even though my .bashrc limits the number of cores. Is there any further config required for 64-bit personal licence?

[dk@server ~]$ cat ~/.bashrc
export PS1='\[\e[1;34m\][\u@\h \w]\$\[\e[m\] '
export PATH=$PATH:/home/quant/q:/home/quant/q/l64
export QHOME=/home/quant/q
export QLIC=/home/quant/q

alias ls='ls --color=auto'
alias q='rlwrap -r numactl --interleave=all --physcpubind=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 q'

[dk@server ~]$ q # starts successfully, as a test
KDB+ 3.5 2017.10.11 Copyright (C) 1993-2017 Kx Systems
l64/ 16(16)core 64532MB dk server 200.100.90.180 EXPIRE 2018.11.02 daniel@email.com KOD #1234567

q)

Client side:

dk@laptop:~$ ssh -i "certificate.pem" -L 8000:localhost:8888 dk@server

Server side:

[dk@server ~]$ jupyter notebook --no-browser --port=8888
[I 04:47:46.433 NotebookApp] JupyterLab beta preview extension loaded from /bin/anaconda3/lib/python3.6/site-packages/jupyterlab
[I 04:47:46.433 NotebookApp] JupyterLab application directory is /bin/anaconda3/share/jupyter/lab
[I 04:47:46.441 NotebookApp] Serving notebooks from local directory: /home/dk
[I 04:47:46.441 NotebookApp] 0 active kernels
[I 04:47:46.441 NotebookApp] The Jupyter Notebook is running at:
[I 04:47:46.441 NotebookApp] https://server:8888/
[I 04:47:46.441 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
'2018.05.31T04:48:42.184 cores
[I 04:48:42.187 NotebookApp] Kernel started: 5fbbd3e0-6e23-4441-967e-d7f320490951
[I 04:48:45.186 NotebookApp] KernelRestarter: restarting kernel (1/5), new random ports
'2018.05.31T04:48:45.195 cores
[I 04:48:48.196 NotebookApp] KernelRestarter: restarting kernel (2/5), new random ports
'2018.05.31T04:48:48.207 cores
[I 04:48:51.207 NotebookApp] KernelRestarter: restarting kernel (3/5), new random ports
'2018.05.31T04:48:51.216 cores
[I 04:48:54.218 NotebookApp] KernelRestarter: restarting kernel (4/5), new random ports
'2018.05.31T04:48:54.228 cores
[W 04:48:57.230 NotebookApp] KernelRestarter: restart failed
[W 04:48:57.230 NotebookApp] Kernel 5fbbd3e0-6e23-4441-967e-d7f320490951 died, removing from map.
[W 04:48:59.014 NotebookApp] Timeout waiting for kernel_info reply from 5fbbd3e0-6e23-4441-967e-d7f320490951
[E 04:48:59.017 NotebookApp] Error opening stream: HTTP 404: Not Found (Kernel does not exist: 5fbbd3e0-6e23-4441-967e-d7f320490951)
jhanna-kx commented 6 years ago

I don't think jupyter will pick up your alias, you can edit kernel.json directly to put in the core limit however

locate the qpk kernel with

jupyter-kernelspec list

then edit kernel.json in the directory listed and add the core limit commands (I just have 8 cores)

{
 "argv": [
  "numactl",
  "--interleave=all",
  "--physcpubind=0,1,2,3,4,5,6,7",
  "q",
  "jupyterq_kernel.q",
  "-cds",
  "{connection_file}"
 ],
 "display_name": "Q 3.5",
 "language": "q",
 "env": {"JUPYTERQ_SERVERARGS":""}
}