Cannot connect to jupyter/ipython instance

[jzuhone]

When attempting to run a jupyter notebook or qtconsole, I get error messages like this:

Invalid argument (bundled/zeromq/src/tcp_address.cpp:171)

The normal ipython prompt works fine.

[sunilmut]

Thanks for trying this out. Could you collect a strace for the failing command to what might be failing?

[chrisgorgo]

I can confirm this bug. You can find the strace here: https://gist.github.com/chrisfilo/2dd2c47592423291bf16db6348d26c6a

[chrisgorgo]

It seems that this bug is caused by https://github.com/jupyter/notebook/issues/1331

[arthurhjorth]

Trying to get the same thing to run too, and with the same result. I'm on a Surface 3 4GB/128GB and happy to aid with testing/pinning things down. Jupyter is such an excellent tool and being able to run this on Bash on Windows would be amazing.

[guydepauw]

Same setup as @arthurhjorth here and also happy to help out. Loving bash on windows so far. Getting jupyter up and running would be the icing on the cake.

[aseering]

I came up with a workaround based on @MadcowD 's investigation on jupyter/notebook#1331 . I submitted it upstream as zeromq/libzmq#2013 , and posted an updated Ubuntu 14.04 package to a PPA in case anyone would like to try to use it.

Standard disclaimers -- I make no guarantees about either security or correctness; if it eats your data or steps on your cat I take no responsibility. But it seems to work for me.

Of course, getting a fix into WSL would be ideal. My cursory understanding of this code is that WSL would need to implement getifaddrs(), and would I think need to at minimum report on the existence of a loopback interface.

[sgraphics]

@aseering can you please explain how to use this fix? I can 'make install' for libzmq from repo source (including this fix) but how do I setup Jupyter to use the custom libzmq build? - it is still giving the same error that is quoted in the original post. Thank you!

[aseering]

@sgraphics are you familiar with Ubuntu PPAs? I would suggest using the PPA that I created that contains the fix. The simplest way to do so would be, I believe:

sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3

That last command would normally install Ubuntu's libzmq; but because you've added my PPA containing the updated package, it will get the updated package instead.

If you want to inspect the source code, you're welcome to download it from the PPA's website (see the link above). You can also follow the first two steps above, then run apt-get source libzmq3, which will download the source code for the package to the current directory. Note that PPAs can contain updated versions of any package; you should only add PPAs that you trust.

Unfortunately, make install will not work in this case: The upstream version that contains the fix is also too new to work on Ubuntu 14.04. The PPA contains the same fix but applied to the source code used by the official Ubuntu 14.04 package. (I downloaded the Ubuntu source using apt-get source, then used quilt to apply the diff containing my fix, and uploaded the updated package.)

[guydepauw]

@aseering this works perfectly. Thanks a million!

[pulinagrawal]

@aseering
I am getting an error 'Invalid argument' in line 190 of tcp_address.cpp even after updating the libzmq3 as you described above. I couldn't build from source because it was asking for source URI when I was trying to download source.

[jzuhone]

I had the same issue as @pulinagrawal.

On Jul 13, 2016, at 11:22 AM, Pulin Agrawal notifications@github.com wrote:

@aseering

I am getting an error 'Invalid argument' in line 190 of tcp_address.cpp even after updating the libzmq3 as you described above. I couldn't build from source because it was asking for source URI when I was trying to download source.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[aseering]

@pulinagrawal @jzuhone -- hm... Could you post the output of dpkg -l libzmq3? It should indicate that you have version 4.0.4+dfsg-2wsl1 installed. (The wsl1 at the end is the key.)

[aseering]

I've posted a bug report about this issue to the Ubuntu libzmq3 bugtracker:

https://bugs.launchpad.net/ubuntu/+source/zeromq3/+bug/1602900

The fix is known; hopefully they will agree to backport it so that WSL users don't have to keep doing this dance to install packages from a PPA.

[mrsiamen]

this fix doesn't work for my Surface 3... I can still see the message Invalid argument (bundled/zeromq/src/tcp_address.cpp:171)

Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-========================-=================-=================-===================================================== ii libzmq3:amd64 4.0.4+dfsg-2wsl1 amd64 lightweight messaging kernel (shared library)

[aseering]

@mrsiamen -- it sounds like you have the new package but it's not getting used for some reason. Is this a clean install of WSL? Did you try installing libzmq3 yourself before using the PPA?

Could you post the results of the following commands?:

ldd /usr/lib/python2.7/dist-packages/zmq/backend/cython/utils.x86_64-linux-gnu.so | grep zmq
md5sum /usr/lib/x86_64-linux-gnu/libzmq.so.3.1.0

For reference, on my machine, I see:

$ ldd /usr/lib/python2.7/dist-packages/zmq/backend/cython/utils.x86_64-linux-gnu.so | grep zmq
        libzmq.so.3 => /usr/lib/x86_64-linux-gnu/libzmq.so.3 (0x00007f72cc2a0000)
$ md5sum /usr/lib/x86_64-linux-gnu/libzmq.so.3
87b97738b28a38a8e89796ecd41862b0  /usr/lib/x86_64-linux-gnu/libzmq.so.3.1.0

[pulinagrawal]

@aseering

on dpkg on libzmq3 I do have wsl1 at the end

ldd gives ldd: /usr/lib/python2.7/dist-packages/zmq/backend/cython/utils.x86_64-linux-gnu.so: No such file or directory

md5sum is the same as your output

[aseering]

@pulinagrawal -- Sounds like you're not using Ubuntu's iPython package? This fix only fixes Ubuntu's libzmq package. If you're using an iPython that pulls in libzmq from elsewhere, they will need to apply the fix as well.

To be specific, I'm testing as follows:

• Uninstall and fully remove any third-party ipython versions that you may have installed
• Run the command sudo apt-get install ipython
• Once ipython has successfully installed, run ipython notebook --no-browser. It should print out a URL. Open a browser in Windows; navigate to that URL.

If you want to track this down further, you could run a command like

find /usr $HOME -type d -name zmq | grep -v IPython

to try to figure out the actual path to the Python zmq module that you're running. Then update the first command above with the appropriate path and re-run ldd; that should tell you where the third-party libzmq that you're using lives.

Another approach, just a guess: Are you getting ipython through Anaconda? I don't personally use Anaconda on Linux (I do on Mac/Windows) because it tends to cause this sort of confusion, and because there's less need for it -- apt-get install and pip install work well on their own.

[mrsiamen]

@aseering Thanks for keep tracking on this:). looks like I'm facing the same issue as pulinagrawal does. Will try to confirm the path.

[mrsiamen]

I try to install the fixed libzmq3 before installing ipython notebook and it works now. the kernel doesn't crash anymore. thank you guy for helping on this :)

libzmq.so.3 => /usr/lib/x86_64-linux-gnu/libzmq.so.3 (0x00007fb3722e0000) 87b97738b28a38a8e89796ecd41862b0 /usr/lib/x86_64-linux-gnu/libzmq.so.3.1.0

[tanguita]

@aseering Could you possibly package this for conda??? unfortunately I have to use a python distribution that is through conda (specific not available packages). Or prehaps tell me how to link it?

[pulinagrawal]

@aseering @mrsiamen

Thank You for the help!! It is fixed now.

I uninstalled ipython and jupyter. I removed the zmq folder from dist-packages. I installed libzmq3 then installed ipython. It did not run yet because I guess removing zmq removed other stuff as well. So, I installed python-zmq. Now it is running.

I might have broken other stuff by just deleting the zmq folder. So I guess better method would have been to use uninstall on python-zmq. But I did not know the name of the package then.

But it works for now.

[aseering]

@tanguita , sorry for the delay -- I believe Anaconda manages their own packages so they would have to incorporate this patch themselves. Feel free to go to Anaconda's bug tracker and file a bug there. If they have questions, I'd be happy to help them with the process.

If you want to get the source code for the patched libzmq, first run the commands above to get the updated package; then run

apt-get source libzmq

This will download the source package from my PPA. You can "cd" into it, read zmq's documentation, and see how to build it.

Alternatively, you can just install the binary package and take the pre-built libzmq that it installs, and try replacing Anaconda's libzmq with Ubuntu's. But that will only work if the two are the same or similar versions; I haven't tested it myself.

[jzuhone]

@aseering, This is not strictly true--one can create a package for Anaconda and upload it to one's own conda channel:

https://conda.anaconda.org/

Then others can install it in their conda distribution.

[tanguita]

This is what I thought but do not know how to do it..

On 21/07/16 11:25, John ZuHone wrote:

@aseering https://github.com/aseering, This is not strictly true--one can create a package for Anaconda and upload it to one's own conda channel:

https://conda.anaconda.org/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Microsoft/BashOnWindows/issues/185#issuecomment-234288922, or mute the thread https://github.com/notifications/unsubscribe-auth/AID11O3GdlLali0cfqZHhW1dxThfSg9mks5qX49tgaJpZM4IG4pi.

[aseering]

Ah, thanks @jzuhone , I didn't realize people can create their own packages. I'll look into that, but I'm a bit busy these days and I'm not a big Anaconda user so I'd have a bunch to learn...

The actual code change to libzmq is quite small; you can see it here. If anyone here knows more about Anaconda packaging, feel free to apply that patch and add it to a channel.

[tanguita]

@jzuhone Is this something you could do or point me on how? I have no idea where to start...

[jzuhone]

@tanguita and others:

I compiled zeromq with the patch as an anaconda package, you can get it by doing:

conda install -c jzuhone zeromq=dev

At best I will keep it updated infrequently, but this should hold everyone over until the stable releases have the fix hopefully.

[jzuhone]

Sorry, found some issues with the above. Try:

conda install -c jzuhone zeromq=4.1.dev0

instead

[adonese]

@aseering I've actually done all of the above. ldd and md5sum are like yours. My problem is that ipython can't detect my python installation. I've installed ipython using sudo apt get install ipython ipython-notebook. And there are also some modules that I had to install using pip. So how do I solve no native kernel is detected issue?

[aseering]

@adonese -- it sounds like your Python package directory has gotten corrupted. You should not be needing to pip install anything to get iPython Notebook working.

Do you have any important state (configuration, files, etc) inside your Bash environment? The simplest solution, though it's a bit heavyweight, would be to simply uninstall it per the instructions here:

https://msdn.microsoft.com/en-us/commandline/wsl/faq

then reinstall it per the instructions here:

https://msdn.microsoft.com/en-us/commandline/wsl/install_guide

and follow the steps above again to install the libzmq fix and install iPython Notebook.

[adonese]

Thanks @aseering. I've finally got it to work. I had to uninstall WSL, the reinstalling it again. IPython is now fully working (It's IPython though, not Jupyter, but I guess the previous will work also on Jupyter).

You also have to install ipython notebook through sudo apt-get install ipython ipython-notebook.

[yasirs]

It's not working for me.

I tried finding my zmq directory like this:

$ find /usr $HOME -type d -name zmq
/usr/lib/python2.7/dist-packages/IPython/kernel/zmq
/usr/local/lib/python2.7/dist-packages/zmq

There is no directory such as /usr/lib/python2.7/dist-packages/zmq

On the other hand:

$ ldd /usr/local/lib/python2.7/dist-packages/zmq/backend/cython/utils.so
        linux-vdso.so.1 =>  (0x00007fffc4f64000)
        libzmq-feeff037.so.5.0.1 => /usr/local/lib/python2.7/dist-packages/zmq/backend/cython/../../.libs/libzmq-feeff037.so.5.0.1 (0x00007f8073273000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8073060000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8072e30000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8072a60000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8072750000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8072430000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8072210000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f8073800000)

md5sum for this is:

$ md5sum /usr/local/lib/python2.7/dist-packages/zmq/backend/cython/../../.libs/libzmq-feeff037.so.5.0.1
cdcd394e58afa01a7c9981ac9cc90c97  /usr/local/lib/python2.7/dist-packages/zmq/backend/cython/../../.libs/libzmq-feeff037.so.5.0.1

But I do have the correct version of libzmq3:

$ dpkg -l libzmq3
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                 Version                 Architecture            Description
+++-====================================-=======================-=======================-==============================================================================
ii  libzmq3:amd64                        4.0.4+dfsg-2wsl1        amd64                   lightweight messaging kernel (shared library)

Can anyone help me understand what's going on?

[aseering]

@yasirs -- you are not using the Ubuntu version of python-zmq. How did you install it?

[yasirs]

@aseering I think I may have installed it using pip. In any case, this is what I did:

$ sudo apt-get install python-zmq
$ sudo -H pip uninstall pyzmq

This solved the problem. Thanks!

[sumedho]

@aseering Thanks for the fix, your changes worked perfectly on a new WSL install. Installed pip, pyhon-dev, scipy stack, ipython, ipython-notebook and then pip install jupyter. Works great! sudo add-apt-repository ppa:aseering/wsl sudo apt-get update sudo apt-get install libzmq3

[RitwikGupta]

Seems like this doesn't work with Python 2.7.12 built from source and linked in via update-alternatives. After sudo apt-get install ipython ipython-notebook, running ipython notebook results in

Traceback (most recent call last):
  File "/usr/bin/ipython", line 5, in <module>
    from IPython import start_ipython
ImportError: No module named IPython

Installing IPython using the pip bundled with that Python distro results in the ipython notebook command working, but then the zeromq error comes up.

[gglanzani]

@RitwikGupta Here's what I did to have it working with the pip version of jupyter.

pip uninstall pyzmq
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3 libzmq3-dev
export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
pip install --no-use-wheel -v pyzmq
pip install jupyter

[mugenZebra]

@gglanzani I tried your method, but this is what I get.

` me@LAPTOP-C55:/mnt/c/Windows/System32$ jupyter notebook

Traceback (most recent call last): File "/usr/local/bin/jupyter-notebook", line 7, in from notebook.notebookapp import main File "/usr/local/lib/python2.7/dist-packages/notebook/notebookapp.py", line 31, in from zmq.eventloop import ioloop ImportError: No module named zmq.eventloop `

[mugenZebra]

I tried sudo apt-get install python-zmq instead and it worked now.

[RitwikGupta]

@gglanzani Your fix worked! Though, running jupyter results in zsh: command not found: jupyter, but that may be due to my local pip installation. ipython notebook --no-browser works perfectly.

import sys
print sys.version_info
>>> sys.version_info(major=2, minor=7, micro=12, releaselevel='final', serial=0)

[vinayprabhu]

@gglanzani This works. Thanks a bunch mate! Now onto getting tensorflow to talk to my sweet GTX960M GPU.

[arthurhjorth]

Thank you so much for the help, everybody. Jupyter is working on my computer now, running on a python3 kernel.

It took me a while to piece together all the individual pieces of advice/directions, because I'm a Linux/apt-get newbie.. So just in case anyone else is confused, hopefully I can help save some time with these instructions:

1. Clean install.
2. This:

sudo apt-get update 
sudo apt-get install python-pip ipython ipython-notebook ipython3 ipython3-notebook 
sudo add-apt-repository ppa:aseering/wsl 
sudo apt-get update 
sudo apt-get install libzmq3 
sudo apt-get install python3-pip 
sudo pip3 install jupyter 
sudo pip install jupyter

Will get it up and running. Some of this may be redundant - I don't know. But it made it work, so figured I would share.

[rdodev]

All, I followed the instructions above and unfortunately things did not work for me.

[W 12:02:11.358 NotebookApp] 404 GET /nbextensions/widgets/notebook/js/extension.js?v=20160822120145 (127.0.0.1) 8.01ms referer=http://localhost:8888/notebooks/Untitled.ipynb?kernel_name=python2
[I 12:02:11.613 NotebookApp] Kernel started: 75e0b050-5e54-4499-86f1-8db750790279
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[I 12:02:14.615 NotebookApp] KernelRestarter: restarting kernel (1/5)
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[I 12:02:17.641 NotebookApp] KernelRestarter: restarting kernel (2/5)
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[W 12:02:21.647 NotebookApp] Timeout waiting for kernel_info reply from 75e0b050-5e54-4499-86f1-8db750790279
[I 12:02:23.664 NotebookApp] KernelRestarter: restarting kernel (1/5)
WARNING:root:kernel 75e0b050-5e54-4499-86f1-8db750790279 restarted
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[I 12:02:26.690 NotebookApp] KernelRestarter: restarting kernel (2/5)
WARNING:root:kernel 75e0b050-5e54-4499-86f1-8db750790279 restarted
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)
[I 12:02:29.727 NotebookApp] KernelRestarter: restarting kernel (3/5)
Invalid argument (bundled/zeromq/src/tcp_address.cpp:190)

Very disappointing. Not sure if the issue lies on the Ubuntu side or the windows side. I don't think Jupyter is directly involved in that issue.

[rdodev]

I want to report that the only way to get this to work was through a clean install. ICYMI:

In a good old windows cmd.exe prompt type: lxrun /uninstall /all It will raise a warning letting you that all data (including your home dir) will be wiped. After that process is done (it should take 2-3 minutes), type: lxrun /install to re-install the ubuntu subsystem from scratch. Once that's done here's what I did:

    1  sudo apt-get update
    2  sudo add-apt-repository ppa:aseering/wsl
    3  sudo apt-get update
    4  sudo apt-get install python-pip ipython ipython-notebook
    5  ipython notebook --no-browser

It now works flawlessly, but with the old iPython UI (which I'm totally OK with given it works)

[allblues1001]

Hi guys, This thread has been very useful for me to set up Jupyter kernel and stuff. I used Windows 10 bash to install it and tensorflow. I run Jupyter in bash and open Windows browser as http://localhost:8888. Jupyter works and my files are there. My intention is to right a tensorflow code and execute it but I get the following errors: ImportError Traceback (most recent call last)

in () ----> 1 import tensorflow as tf 2 import numpy as np ImportError: No module named tensorflow What can be the possible reason?

[rdodev]

@allblues1001 can you run python in a bash prompt, then when you get the python REPL prompt >>> type import tensorflow and let us know the result of that is?

[allblues1001]

@rdodev Thanks for the response. Yes, I am able to run python and import tensorflow in the bash prompt. Absolutely no problem in that. It is only when I use Windows browser (Google Chrome), it fails to import tensorflow, numpy etc. Interestingly, the python commands are working fine like adding two numbers etc. It is only when I try to import some library like tensorflow and numpy, I get these errors. Can there be some communication problem?

[RitwikGupta]

Did you install TensorFlow via pip? Could you post the result of python --version and pip --version?

On Wed, Aug 24, 2016 at 9:01 AM -0400, "allblues1001" notifications@github.com<mailto:notifications@github.com> wrote:

@rdodevhttps://github.com/rdodev Thanks for the response. Yes, I am able to run python and import tensorflow in the bash prompt. Absolutely no problem in that. It is only when I use Windows browser (Google Chrome), it fails to import tensorflow, numpy etc. Interestingly, the python commands are working fine like adding two numbers etc. It is only when I try to import some library like tensorflow and numpy, I get these errors. Can there be some communication problem?

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Microsoft/BashOnWindows/issues/185#issuecomment-242054527, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEurpdNlSNKjKTTqy95y2BASscw6s3PJks5qjEChgaJpZM4IG4pi.

[allblues1001]

@RitwikGupta : Yes, I used pip to install Tensorflow. The results are: Python 2.7.6 and pip 1.5.4

[RitwikGupta]

Could you please also list the complete path of the installation of Python and pip? Just confirming that you're not using a version of pip provided by a separate Python installation and then launching ipython notebook provided by apt.

On Wed, Aug 24, 2016 at 9:15 AM -0400, "allblues1001" notifications@github.com<mailto:notifications@github.com> wrote:

@RitwikGuptahttps://github.com/RitwikGupta : Yes, I used pip to install Tensorflow. The results are: Python 2.7.6 and pip 1.5.4

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Microsoft/BashOnWindows/issues/185#issuecomment-242058810, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEurpcxvr_fulTt1ABh2dcis29rFqUrLks5qjEPKgaJpZM4IG4pi.