behavior of findAll does not match `jupyter kernelspec list` on a macOS.

[williamstein]

I'm not using homebrew or Macports or anything funny; this is a vanilla macOS Big Sur (11.1, Apple Silicon). I did

pip3 install --user jupyter jupyter-console

Then

wstein@Williams-MBP jupyter % jupyter kernelspec list
Available kernels:
  python3    /Users/wstein/Library/Python/3.8/lib/python/site-packages/ipykernel/resources

Of course, look at the "interesting" directory the kernel spec is in above.

Now using the kernelspecs library shows no kernels:

> async function f() { console.log(await (require('kernelspecs').findAll)())}
undefined
> f()
Promise { <pending> }
> {}

This isn't surprising giving the source code in traverse.js of this library:

function findAll() {
  return jp.dataDirs({ withSysPrefix: true }).then(dirs => {
    return Promise.all(dirs
      // get kernel infos for each directory and ignore errors
      .map(dir => getKernelInfos(path.join(dir, 'kernels')).catch(() => {}))
    ).then(extractKernelResources)
  });
}

Here jp.dataDirs has /Users/wstein/Library/Python/ in it, but nothing else in my home directory. So basically, you assume that the kernels are all in foo/kernels for some foo in the output of dataDirs. So either dataDirs is wrong or this assumption is wrong, where by "wrong" I mean "different than jupyter kernelspec list".

Note also that jupyter console --kernel=python3 does work fine.

WORKAROUND: I'll probably just copy that kernel spec file somewhere else for now to work around this.

[williamstein]

Wow, all that resources path has in it is two png images:

wstein@Williams-MBP jupyter % ls /Users/wstein/Library/Python/3.8/lib/python/site-packages/ipykernel/resources
logo-32x32.png  logo-64x64.png

I don't claim to understand jupyter kernelspec list...

The actual kernel spec is here:

/Users/wstein/Library/Python/3.8/share/jupyter/kernels/python3

In any case, workaround:

bash-3.2$ cd /Users/wstein/Library/Jupyter/
bash-3.2$ ln -s /Users/wstein/Library/Python/3.8/share/jupyter/kernels .
bash-3.2$ pwd
/Users/wstein/Library/Jupyter
bash-3.2$ ls -lht
total 0
lrwxr-xr-x  1 wstein  staff    54B Dec 30 12:54 kernels -> /Users/wstein/Library/Python/3.8/share/jupyter/kernels

Then it works correctly:

wstein@Williams-MBP src % pwd
/Users/wstein/build/cocalc/src
wstein@Williams-MBP src % cd smc-project/jupyter
wstein@Williams-MBP jupyter % node
Welcome to Node.js v15.5.0.
Type ".help" for more information.
> async function f() { console.log(await (require('kernelspecs').findAll)())}
undefined
> f()
Promise { <pending> }
> [
  '/Users/wstein/Library/Jupyter',
  '/usr/local/share/jupyter',
  '/usr/share/jupyter'
]
{
  python3: {
    name: 'python3',
    files: [
      '/Users/wstein/Library/Jupyter/kernels/python3/kernel.json',
      '/Users/wstein/Library/Jupyter/kernels/python3/logo-32x32.png',
      '/Users/wstein/Library/Jupyter/kernels/python3/logo-64x64.png'
    ],
    resources_dir: '/Users/wstein/Library/Jupyter/kernels/python3',
    spec: { argv: [Array], display_name: 'Python 3', language: 'python' }
  }
}

[williamstein]

How to see exactly what dataDirs is:

async function g() { console.log(await (require('jupyter-paths').dataDirs)())}; g()