Jupyter attempts to run previous version of node after upgrade

thisgeek commented 8 years ago

After upgrading my node installation, running a javascript notebook fails to start up the kernel. I had previously registered iJavascript with the dashboard for all users as advised in the usage notes. I get this error:

Failed to run command:
    [u'/usr/local/Cellar/node/5.9.0/bin/node', u'/usr/local/lib/node_modules/ijavascript/lib/kernel.js', u'/Users/cjoslyn/Library/Jupyter/runtime/kernel-7e18a9a6-3112-4884-99ad-7804cbb2347f.json', u'--protocol=5.0']

I can see from the error's stacktrace that this is due to a missing file:

OSError: [Errno 2] No such file or directory

/usr/local/Cellar/node/5.9.0/bin/node no longer exists on my machine after I upgraded to 5.9.1. I was able to resolve this temporarily by creating a symlink so that /usr/local/Cellar/node/5.9.1 points to /usr/local/Cellar/node/5.9.0. I don't know what would resolve this permanently.

I thought that I might be able to resolve by re-registering the kernel. I ran ijs --ijs-install=global, but it appears to have on effect on the issue.

Environment data:

$ ijs --version
5.0.11-beta.0
$ npm version
{ npm: '3.7.3',
  ares: '1.10.1-DEV',
  http_parser: '2.6.2',
  icu: '56.1',
  modules: '47',
  node: '5.9.1',
  openssl: '1.0.2g',
  uv: '1.8.0',
  v8: '4.6.85.31',
  zlib: '1.2.8' }
$ ipython --version
4.1.2

The operating system is Mac OS X 10.11.3.

n-riesco commented 8 years ago

Please, could you post the output of running ipython kernelspec list --json before and after running ijs --ijs-install=global?

thisgeek commented 8 years ago

Sure.

Before

$ ipython kernelspec list --json
[TerminalIPythonApp] WARNING | Subcommand `ipython kernelspec` is deprecated and will be removed in future versions.
[TerminalIPythonApp] WARNING | You likely want to use `jupyter kernelspec`... continue in 5 sec. Press Ctrl-C to quit now.
{
  "kernelspecs": {
    "javascript": {
      "resource_dir": "/Users/cjoslyn/Library/Jupyter/kernels/javascript",
      "spec": {
        "language": "javascript",
        "display_name": "Javascript (Node.js)",
        "env": {},
        "argv": [
          "/usr/local/Cellar/node/5.9.0/bin/node",
          "/usr/local/lib/node_modules/ijavascript/lib/kernel.js",
          "{connection_file}",
          "--protocol=5.0"
        ]
      }
    },
    "bash": {
      "resource_dir": "/Users/cjoslyn/Library/Jupyter/kernels/bash",
      "spec": {
        "language": "bash",
        "display_name": "Bash",
        "env": {
          "PS1": "$"
        },
        "argv": [
          "/usr/local/opt/python/bin/python2.7",
          "-m",
          "bash_kernel",
          "-f",
          "{connection_file}"
        ]
      }
    },
    "python2": {
      "resource_dir": "/usr/local/lib/python2.7/site-packages/ipykernel/resources",
      "spec": {
        "language": "python",
        "display_name": "Python 2",
        "env": {},
        "argv": [
          "/usr/local/opt/python/bin/python2.7",
          "-m",
          "ipykernel",
          "-f",
          "{connection_file}"
        ]
      }
    }
  }
}

After

$ ipython kernelspec list --json
[TerminalIPythonApp] WARNING | Subcommand `ipython kernelspec` is deprecated and will be removed in future versions.
[TerminalIPythonApp] WARNING | You likely want to use `jupyter kernelspec`... continue in 5 sec. Press Ctrl-C to quit now.
{
  "kernelspecs": {
    "javascript": {
      "resource_dir": "/Users/cjoslyn/Library/Jupyter/kernels/javascript",
      "spec": {
        "language": "javascript",
        "display_name": "Javascript (Node.js)",
        "env": {},
        "argv": [
          "/usr/local/Cellar/node/5.9.0/bin/node",
          "/usr/local/lib/node_modules/ijavascript/lib/kernel.js",
          "{connection_file}",
          "--protocol=5.0"
        ]
      }
    },
    "bash": {
      "resource_dir": "/Users/cjoslyn/Library/Jupyter/kernels/bash",
      "spec": {
        "language": "bash",
        "display_name": "Bash",
        "env": {
          "PS1": "$"
        },
        "argv": [
          "/usr/local/opt/python/bin/python2.7",
          "-m",
          "bash_kernel",
          "-f",
          "{connection_file}"
        ]
      }
    },
    "python2": {
      "resource_dir": "/usr/local/lib/python2.7/site-packages/ipykernel/resources",
      "spec": {
        "language": "python",
        "display_name": "Python 2",
        "env": {},
        "argv": [
          "/usr/local/opt/python/bin/python2.7",
          "-m",
          "ipykernel",
          "-f",
          "{connection_file}"
        ]
      }
    }
  }
}

n-riesco commented 8 years ago

I'm surprised that ijs --ijs-install=global doesn't print out any error messages. Let's try to enable the debug messages.

Could you install the debug package:

npm install -g debug

and post what you get when you run:

$ DEBUG=IJS:* ijs --ijs-install=global

thisgeek commented 8 years ago

$ DEBUG=IJS:* ijs --ijs-install=global
  IJS: CONTEXT: +0ms { path:
   { node: '/usr/local/Cellar/node/5.9.1/bin/node',
     root: '/usr/local/lib/node_modules/ijavascript',
     kernel: '/usr/local/lib/node_modules/ijavascript/lib/kernel.js',
     images: '/usr/local/lib/node_modules/ijavascript/images' },
  packageJSON:
   { _args: [ [Object] ],
     _from: 'ijavascript@latest',
     _id: 'ijavascript@5.0.11-beta.0',
     _inCache: true,
     _installable: true,
     _location: '/ijavascript',
     _npmOperationalInternal:
      { host: 'packages-13-west.internal.npmjs.com',
        tmp: 'tmp/ijavascript-5.0.11-beta.0.tgz_1457186984776_0.005411402089521289' },
     _npmUser: { email: 'enquiries@nicolasriesco.net', name: 'n-riesco' },
     _npmVersion: '1.4.21',
     _phantomChildren: {},
     _requested:
      { name: 'ijavascript',
        raw: 'ijavascript',
        rawSpec: '',
        scope: null,
        spec: 'latest',
        type: 'tag' },
     _requiredBy: [ '#USER' ],
     _resolved: 'https://registry.npmjs.org/ijavascript/-/ijavascript-5.0.11-beta.0.tgz',
     _shasum: 'b695d7ffed6e658bdd9eb18642c26fda253a754f',
     _shrinkwrap: null,
     _spec: 'ijavascript',
     _where: '',
     author:
      { email: 'enquiries@nicolasriesco.net',
        url: 'http://www.nicolasriesco.net/' },
     bin: { ijs: 'bin/ijavascript.js' },
     bugs: { url: 'https://github.com/n-riesco/ijavascript/issues' },
     contributors: [ [Object], [Object], [Object], [Object], [Object], [Object] ],
     dependencies: { jmp: '0.4.x', nel: '>=0.4.0-beta.0', 'node-uuid': '^1.4.2' },
     description: 'IJavascript is a Javascript kernel for the Jupyter notebook',
     devDependencies: { jsdoc: 'latest', jshint: 'latest' },
     directories: {},
     dist:
      { shasum: 'b695d7ffed6e658bdd9eb18642c26fda253a754f',
        tarball: 'https://registry.npmjs.org/ijavascript/-/ijavascript-5.0.11-beta.0.tgz' },
     gitHead: '67054ede7b8889133c1e46d837d4c67ca08ef085',
     homepage: 'https://n-riesco.github.io/ijavascript',
     keywords: [ 'javascript', 'kernel', 'ipython', 'jupyter' ],
     license: 'BSD-3-Clause',
     maintainers: [ [Object] ],
     name: 'ijavascript',
     optionalDependencies: {},
     readme: 'ERROR: No README data found!',
     repository:
      { type: 'git',
        url: 'git+https://github.com/n-riesco/ijavascript.git' },
     scripts:
      { doc: 'python scripts/doc-build.py',
        'doc:publish': 'node scripts/doc-publish.js gh-pages https://github.com/n-riesco/ijavascript',
        lint: 'jshint bin lib',
        test: 'node test/index.js',
        'test:4': 'node test/index.js --protocol=4.1',
        'test:5': 'node test/index.js --protocol=5.0' },
     version: '5.0.11-beta.0' },
  flag: { install: 'global' },
  args:
   { kernel:
      [ '/usr/local/Cellar/node/5.9.1/bin/node',
        '/usr/local/lib/node_modules/ijavascript/lib/kernel.js',
        '{connection_file}',
        '--protocol=5.0' ],
     frontend: [ 'ipython', 'notebook' ] },
  protocol: { version: '5.0', majorVersion: 5 },
  frontend: { version: '4.1.2', majorVersion: 4 } }

n-riesco commented 8 years ago

This bit:

args:
   { kernel:
      [ '/usr/local/Cellar/node/5.9.1/bin/node',
        '/usr/local/lib/node_modules/ijavascript/lib/kernel.js',
        '{connection_file}',
        '--protocol=5.0' ],
     frontend: [ 'ipython', 'notebook' ] },

means that ijs sees the new Node installation.

Is it possible that IJavascript is also installed locally?

If I'm not mistaken IPython v4.1.2 doesn't provide the option to uninstall a kernel (jupyter kernelspec remove --user javascript).

Could you check whether a javascript kernel is installed locally:

ls $(ipython locate)/kernels

thisgeek commented 8 years ago

$ ls `ipython locate`/kernels
ls: /Users/cjoslyn/.ipython/kernels: No such file or directory

It does not appear to be installed locally. The issue is still present. If I remove the symlink I still get a kernel failure.

n-riesco commented 8 years ago

It looks like Jupyter may store the local kernels somewhere else now. See:

user@host:~$ ls $(jupyter --data-dir)/kernels
javascript

I cannot reproduce this issue on a VM using the same Jupyter version. I think the easiest explanation is that IJavascript was installed locally in the past. Could you check if the above command works for you?

thisgeek commented 8 years ago

I can't account for how iJavascript may have been installed locally in the past. I haven't used iJavascript until very recently.

thisgeek commented 8 years ago

The command works. Here's the way it looks when I use tree.

$ tree $(jupyter --data-dir)/kernels
/Users/cjoslyn/Library/Jupyter/kernels
├── bash
│   └── kernel.json
└── javascript
    ├── kernel.json
    ├── logo-32x32.png
    └── logo-64x64.png

2 directories, 4 files

n-riesco commented 8 years ago

OK, that looks like a local installation of a kernel. You should be able to uninstall it by running:

npm uninstall ijavascript

thisgeek commented 8 years ago

I was not able to uninstall. I don't have a local installation of the ijavascript module.

rgbkrk commented 8 years ago

Every time I upgrade node (via nvm) I end up having to nuke the kernel directory for ijavascript and installing again.

c22 commented 8 years ago

Not sure what causes this either, but the workaround for me without having to nuke/pave ijs each time was to update argv in kernel.json to point to /usr/local/bin/node (a symlink to my most current homebrew node installation) instead of the specific node version that it was installed with.

n-riesco commented 8 years ago

@c22 I think this issue may be caused by #75 . Could you check whether replacing ipython with jupyter in this line fixes the problem?

n-riesco commented 8 years ago

@c22 @thisgeek I've just released v5.0.12 with a fix to use Jupyter to install the IJavascript kernel. If still possible, please, could you check whether the latest release fixes this issue for you?

c22 commented 8 years ago

Thanks, I'll keep an eye on it when I upgrade node next.

c22 commented 8 years ago

Upgrading from node 6.5.0 to 6.7.0 via homebrew resulted in the same issue, kernel.json has the symlink target of "/usr/local/Cellar/node/6.5.0/bin/node" hardcoded instead of the symlink "/usr/local/bin/node".

n-riesco commented 8 years ago

On 29/09/16 07:45, c22 wrote:

Upgrading from node 6.5.0 to 6.7.0 via homebrew resulted in the same issue, kernel.json has the symlink target of |"/usr/local/Cellar/node/6.5.0/bin/node"| hardcoded instead of the symlink |"/usr/local/bin/node"|.

Could you post the output of running the following?

$ node -e "console.log(process.argv)"
[ '/usr/bin/nodejs' ]

$ ijs --version
5.0.14

$ jupyter kernelspec list --json
...

c22 commented 8 years ago

c22@machine ~> node -e 'console.log(process.argv)'
[ '/usr/local/Cellar/node/6.7.0/bin/node' ]
c22@machine ~> ijs --version
5.0.14
c22@machine ~> jupyter kernelspec list --json
{
  "kernelspecs": {
    "javascript": {
      "resource_dir": "/Users/c22/Library/Jupyter/kernels/javascript",
      "spec": {
        "language": "javascript",
        "display_name": "Javascript (Node.js)",
        "argv": [
          "/usr/local/bin/node",
          "/usr/local/lib/node_modules/ijavascript/lib/kernel.js",
          "{connection_file}",
          "--protocol=5.0"
        ],
        "env": {}
      }
    },
    "julia-0.4": {
      "resource_dir": "/Users/c22/Library/Jupyter/kernels/julia-0.4",
      "spec": {
        "language": "julia",
        "display_name": "Julia 0.4.6",
        "argv": [
          "/Applications/Julia-0.4.6.app/Contents/Resources/julia/bin/julia",
          "-i",
          "--startup-file=yes",
          "/Users/c22/.julia/v0.4/IJulia/src/kernel.jl",
          "{connection_file}"
        ],
        "env": {}
      }
    },
    "python3": {
      "resource_dir": "/usr/local/lib/python3.5/site-packages/ipykernel/resources",
      "spec": {
        "language": "python",
        "display_name": "Python 3",
        "argv": [
          "/usr/local/opt/python3/bin/python3.5",
          "-m",
          "ipykernel",
          "-f",
          "{connection_file}"
        ],
        "env": {}
      }
    }
  }
}
c22@machine ~> which node
/usr/local/bin/node

So the problem is likely obvious when you see output of the first command. Interestingly it looks like IJulia would probably have a similar issue.

c22 commented 8 years ago

I don't know a nice way around this, could you check for the output of command -s node if the platform is POSIX and use that instead of process.argv?

The approach IJulia has taken is (sadly)

if you install a new Julia binary (or do anything that changes the location of Julia on your computer), you must update the IJulia installation

n-riesco commented 7 years ago

Fixed in the latest release IJavascript@5.0.15

n-riesco / ijavascript

Jupyter attempts to run previous version of node after upgrade #67

Before

After