Closed maresb closed 3 years ago
@phisanti, my solution is very particular to the complicated configuration I'm running. The first step is to look at the output via the method I explained above.
In case you want to run your own Jupyter server, you can run jupyter lab
from the command line to start JupyterLab. After JupyterLab starts, at the bottom of the terminal where you started JupyterLab you should see:
To access the server, open this file in a browser:
file:///home/jovyan/.local/share/jupyter/runtime/jpserver-32688-open.html
Or copy and paste one of these URLs:
http://13ca0fa98ac2:8889/lab?token=7ed6ea0f36651733bc695a24eae12700938f44556978b9aa
or http://127.0.0.1:8889/lab?token=7ed6ea0f36651733bc695a24eae12700938f44556978b9aa
You can take this URL http://127.0.0.1:8889/lab?token=7ed6ea0f36651733bc695a24eae12700938f44556978b9aa
and put it into your configuration. Then the MS Python extension will (hopefully) be able to connect to your Jupyter server.
Thanks for opening up a separate issue @maresb! In order for us to look into this further, do you mind updating the issue with the following:
Thank you!
@jsjoeio thanks for being so eager to debug this issue! I have not been so forthcoming with details due to the complicated nature of my setup. (See the second paragraph which I added below the horizontal rule.) I hope this explains why I don't explain how to reproduce, since that would involve setting up the Jupyter server proxy.
I recommend waiting for @phisanti to arrive and debugging his particular problem, since I suspect that his setup is more straightforward than mine. (Moreover, I have a solution for my particular case, so I opened this issue mainly for the benefit of him and those who also have this issue.)
My suspicion is that prior to code 3.9, MS Python was somehow proxying its own Jupyter server, but that from 3.9 the connection is no longer being proxied. But I don't really know so many details.
I prepared the image like you did use docker. Then, build the machine in a remote machine on paperspace.com notebook, which uses port 8888 to push the vscode server to the browser.
So you can install the image from https://hub.docker.com/repository/docker/phisanti/gradient-coder. The entry point is /run.sh
. Next, I just download and install 'ms-python.python-2020.10.332292344':
curl -JLO https://open-vsx.org/api/ms-python/python/2020.10.332292344/file/ms-python.python-2020.10.332292344.vsix
code-server --install-extension ms-python.python-2020.10.332292344'.vsix
Next, I will install the required python dependencies as I am asked. My final lib looks like:
# packages in environment at /home/coder/miniconda3:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
argon2-cffi 20.1.0 py38h27cfd23_1
astroid 2.5 py38h06a4308_1
async_generator 1.10 pyhd3eb1b0_0
attrs 20.3.0 pyhd3eb1b0_0
backcall 0.2.0 pyhd3eb1b0_0
beautifulsoup4 4.9.3 pyha847dfd_0
bleach 3.3.0 pyhd3eb1b0_0
brotlipy 0.7.0 py38h27cfd23_1003
bzip2 1.0.8 h7b6447c_0
ca-certificates 2021.1.19 h06a4308_1
certifi 2020.12.5 py38h06a4308_0
cffi 1.14.3 py38h261ae71_2
chardet 3.0.4 py38h06a4308_1003
conda 4.9.2 py38h06a4308_0
conda-build 3.21.4 py38h06a4308_0
conda-package-handling 1.7.2 py38h03888b9_0
cryptography 3.2.1 py38h3c74f83_1
dbus 1.13.18 hb2f20db_0
decorator 4.4.2 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
entrypoints 0.3 py38_0
expat 2.2.10 he6710b0_2
filelock 3.0.12 pyhd3eb1b0_1
fontconfig 2.13.1 h6c09931_0
freetype 2.10.4 h5ab3b9f_0
glib 2.67.4 h36276a3_1
glob2 0.7 pyhd3eb1b0_0
gst-plugins-base 1.14.0 h8213a91_2
gstreamer 1.14.0 h28cd5cc_2
icu 58.2 he6710b0_3
idna 2.10 py_0
importlib-metadata 2.0.0 py_1
importlib_metadata 2.0.0 1
ipykernel 5.3.4 py38h5ca1d4c_0
ipython 7.21.0 py38hb070fc8_0
ipython_genutils 0.2.0 pyhd3eb1b0_1
ipywidgets 7.6.3 pyhd3eb1b0_1
isort 5.7.0 pyhd3eb1b0_0
jedi 0.17.0 py38_0
jinja2 2.11.3 pyhd3eb1b0_0
jpeg 9b h024ee3a_2
json5 0.9.5 py_0
jsonschema 3.2.0 py_2
jupyter 1.0.0 py38_7
jupyter_client 6.1.7 py_0
jupyter_console 6.2.0 py_0
jupyter_core 4.7.1 py38h06a4308_0
jupyterlab 2.2.6 py_0
jupyterlab_pygments 0.1.2 py_0
jupyterlab_server 1.2.0 py_0
jupyterlab_widgets 1.0.0 pyhd3eb1b0_1
lazy-object-proxy 1.5.2 py38h27cfd23_0
ld_impl_linux-64 2.33.1 h53a641e_7
libarchive 3.4.2 h62408e4_0
libedit 3.1.20191231 h14c3975_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
liblief 0.10.1 he6710b0_0
libpng 1.6.37 hbc83047_0
libsodium 1.0.18 h7b6447c_0
libstdcxx-ng 9.1.0 hdf63c60_0
libuuid 1.0.3 h1bed415_2
libxcb 1.14 h7b6447c_0
libxml2 2.9.10 hb55368b_3
lz4-c 1.9.3 h2531618_0
markupsafe 1.1.1 py38h7b6447c_0
mccabe 0.6.1 py38_1
mistune 0.8.4 py38h7b6447c_1000
nbclient 0.5.3 pyhd3eb1b0_0
nbconvert 6.0.7 py38_0
nbformat 5.1.2 pyhd3eb1b0_1
ncurses 6.2 he6710b0_1
nest-asyncio 1.5.1 pyhd3eb1b0_0
notebook 6.2.0 py38h06a4308_0
openssl 1.1.1j h27cfd23_0
packaging 20.9 pyhd3eb1b0_0
pandoc 2.11 hb0f4dca_0
pandocfilters 1.4.3 py38h06a4308_1
parso 0.8.1 pyhd3eb1b0_0
patchelf 0.12 h2531618_1
pcre 8.44 he6710b0_0
pexpect 4.8.0 pyhd3eb1b0_3
pickleshare 0.7.5 pyhd3eb1b0_1003
pip 20.2.4 py38h06a4308_0
pkginfo 1.7.0 py38h06a4308_0
prometheus_client 0.9.0 pyhd3eb1b0_0
prompt-toolkit 3.0.8 py_0
prompt_toolkit 3.0.8 0
psutil 5.8.0 py38h27cfd23_1
ptyprocess 0.7.0 pyhd3eb1b0_2
py-lief 0.10.1 py38h403a769_0
pycosat 0.6.3 py38h7b6447c_1
pycparser 2.20 py_2
pygments 2.8.1 pyhd3eb1b0_0
pylint 2.7.2 py38h06a4308_1
pyopenssl 19.1.0 pyhd3eb1b0_1
pyparsing 2.4.7 pyhd3eb1b0_0
pyqt 5.9.2 py38h05f1152_4
pyrsistent 0.17.3 py38h7b6447c_0
pysocks 1.7.1 py38h06a4308_0
python 3.8.5 h7579374_1
python-dateutil 2.8.1 pyhd3eb1b0_0
python-libarchive-c 2.9 pyhd3eb1b0_0
pytz 2021.1 pyhd3eb1b0_0
pyyaml 5.4.1 py38h27cfd23_1
pyzmq 20.0.0 py38h2531618_1
qt 5.9.7 h5867ecd_1
qtconsole 5.0.2 pyhd3eb1b0_0
qtpy 1.9.0 py_0
readline 8.0 h7b6447c_0
requests 2.24.0 py_0
ripgrep 12.1.1 0
ruamel_yaml 0.15.87 py38h7b6447c_1
send2trash 1.5.0 pyhd3eb1b0_1
setuptools 50.3.1 py38h06a4308_1
sip 4.19.13 py38he6710b0_0
six 1.15.0 py38h06a4308_0
soupsieve 2.2 pyhd3eb1b0_0
sqlite 3.33.0 h62c20be_0
terminado 0.9.2 py38h06a4308_0
testpath 0.4.4 pyhd3eb1b0_0
tk 8.6.10 hbc83047_0
toml 0.10.1 py_0
tornado 6.1 py38h27cfd23_0
tqdm 4.51.0 pyhd3eb1b0_0
traitlets 5.0.5 pyhd3eb1b0_0
urllib3 1.25.11 py_0
wcwidth 0.2.5 py_0
webencodings 0.5.1 py38_1
wheel 0.35.1 pyhd3eb1b0_0
widgetsnbextension 3.5.1 py38_0
wrapt 1.12.1 py38h7b6447c_1
xz 5.2.5 h7b6447c_0
yaml 0.2.5 h7b6447c_0
zeromq 4.3.3 he6710b0_3
zipp 3.4.0 pyhd3eb1b0_0
zlib 1.2.11 h7b6447c_3
zstd 1.4.5 h9ceee32_0
Finally, I try to create a jupyter notebook, but I got the following error:
Failed to connect to Jupyter notebook. http://127.0.0.1:8889/ Error: Kernel Python 3 is not usable. Check the Jupyter output tab for more information.
If I follow the advice of @maresb and I run jupyter lab from the terminal, then the following the response is:
[I 22:10:09.745 LabApp] The port 8888 is already in use, trying another port.
[I 22:10:09.755 LabApp] JupyterLab extension loaded from /home/coder/miniconda3/lib/python3.8/site-packages/jupyterlab
[I 22:10:09.756 LabApp] JupyterLab application directory is /home/coder/miniconda3/share/jupyter/lab
[C 22:10:09.760 LabApp] Running as root is not recommended. Use --allow-root to bypass.
If I click on the palette command the option to create a new jupyter notebook with the token of my notebook, then the error looks like:
Failed to connect to remote Jupyter notebook. Check that the Jupyter Server URI setting has a valid running server specified. http://127.0.0.1:8889/token=11cf752079724762b4622643143ddb4a FetchError: request to http://127.0.0.1:8889/token=11cf752079724762b4622643143ddb4a/hub/api failed, reason: connect ECONNREFUSED 127.0.0.1:8889
If I use the token you posted, this is the error:
Failed to connect to remote Jupyter notebook. Check that the Jupyter Server URI setting has a valid running server specified. http://127.0.0.1:8889/lab Error: Kernel Python 3 is not usable. Check the Jupyter output tab for more information.
running on:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"
On a final note, when I install ms-python.python-2020.3.69010
, the problem seem to be gone. However, I cannot use widgets (I need nglview to display protein structures). So it seems to be an issue of that particular version with code-server or vscode.
Thanks for the details @phisanti! Sounds like yours might be complex as well. Any chance you can edit your comment and outline it in a way that anyone could jump in and follow? i.e.
The more minimal the steps to reproduce, the faster we'll be able to look into this! If you need inspiration, this article has good tips.
@phisanti Does it work if you publish port 8889 from Docker?
@maresb I have edited the entrypoint. Now it looks like:
#!/bin/bash
# Source: https://gist.github.com/earthgecko/3089509
# Generates a random alphanumeric string of length 48 (like a jupyter notebook token i.e. c8de56fa4deed24899803e93c227592aef6538f93025fe01)
if [ -z "$JUPYTER_TOKEN" ]; then
JUPYTER_TOKEN=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 48 | head -n 1)
fi
# Note: print mocked jupyter token so that we can run this container as if it is a notebook within Gradient V1
echo "http://localhost:8889/?token=${JUPYTER_TOKEN}"
echo "http://localhost:8889/\?token\=${JUPYTER_TOKEN}"
PASSWORD=${JUPYTER_TOKEN} /usr/bin/code-server --bind-addr "0.0.0.0:8889" .
However, I cannot access the session with this configuration.
@phisanti What does your docker
command look like? Did you try adding -p 8889:8889
?
I don't quite understand what is the goal of editing the entrypoint. Are you trying to change the code-server
port to 8889?
What I meant with my last suggestion is that you should keep code-server
on 8888, but also publish port 8889 so that the browser running code-server
can access the Jupyter server as well.
So I recommend undoing the change to your entrypoint. In your docker
command you probably have something like -p 8888:8888
and my suggestion would be to change this to -p 8888:8888 -p 8889:8889
.
I might look like a fool because I am new to docker
and all programming related. My major is in biology and I need to run some molecular dynamics simulations.
This is how my docker file looks now:
# Import image with code-server and CUDA
FROM phisanti/cuda-code-server:latest
# Install conda
RUN curl -o ~/miniconda.sh -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
chmod +x ~/miniconda.sh && \
~/miniconda.sh -b && \
rm ~/miniconda.sh && \
/home/coder/miniconda3/bin/conda install conda-build
ENV PATH=$PATH:/home/coder/miniconda3/bin/
# Entrypoint
COPY run.sh /run.sh
EXPOSE 8888
EXPOSE 8889
ENTRYPOINT ["/run.sh"]
You can see with all detail here. I tried to expose the port with the docker command EXPOSE
as I did with 8888
(taken from the template). However, from your comment, I see the -p
flag, so I deduce it should run as RUN code-server -p 8888:8888 -p 8889:8889
, but I am not fully sure. Should I say that using the current image, I still get the same error Failed to connect to Jupyter notebook. http://127.0.0.1:8889/ Error: Kernel Python 3 is not usable.
. I notice that it also says click on OUTPUT for more detail but nothing comes there. Is there an option to see such an output?
Sorry if it is a bad time, I have been struggling with this for at least a month.
Hi @phisanti, my apologies if I came across as condescending. You should feel welcome regardless of your background or technical expertise, so I'm sorry if I failed there. It's great that you're learning Docker, and of course I was completely lost myself as I was a beginner.
I think I know more or less what the problem is, but I was missing some info I which I need to be able to help.
What I am missing is your docker run
command (or your docker-compose.yml
in case you are starting the container with docker-compose
). The -p
is not for code-server
but rather docker run
. So the command should look like:
docker run -p 8888:8888 -p 8889:8889 …
Could you send me the full docker run
command you're using?
By the way, the EXPOSE
command in Dockerfile
is effectively documentation; it doesn't actually do anything. According to the documentation
The
EXPOSE
instruction does not actually publish the port. It functions as a type of documentation between the person who builds the image and the person who runs the container, about which ports are intended to be published. To actually publish the port when running the container, use the-p
flag ondocker run
to publish and map one or more ports, or the-P
flag to publish all exposed ports and map them to high-order ports.
Personally I find this weird, confusing and unexpected on Docker's part. And as the documentation states, we need to figure out what's going on with the -p
flags in your docker run
command.
To explain where I'm going with this, it looks to me like a Jupyter server is being started inside of your Docker container. Then code-server tries to connect to the Jupyter server from your browser. However, the Jupyter server is not running on your native OS, but inside your Docker container. Your browser is looking on your native OS and not finding anything, and thus giving an error.
My suspicion is that if you publish port 8889 (the port where your Jupyter server is starting) from Docker to your native OS, then your browser will be able to find the Jupyter server, and the error will go away.
Does this make sense? Let me know if anything I wrote was confusing.
So, you say that I need to run a command in the terminal like this before publish/build the image:
docker run -p 8888:8888 -p 8889:8889 …
However, this is probably the result of me building the docker image based on youtube tutorials and cut-and-paste templates. I remember I tried to download the docker software in my computer and then build an image, but I end up nowhere. Then, I saw that it was possible to create an image only through writing something called a DOCKERFILE
. Thus, I copied the code-server docker image that paperspace uploaded to their github and made some changes (add CUDA and conda), which resulted in the code I sent previously. So, I discovered that I only need to link my docker account and, voilá, the image appeared in dockerhub. Since I am not using the terminal, then I guess you suggest to add a file called docker-compose.yml
to the github repository? If that is correct, then, I can do it right now. I will google how to organise the file and then add that line on the right spot.
Once again, thank you @maresb for going the extra mile, I really feel you are trying to help me.
before publish/build the image
No, after.
The Docker build process generates from the Dockerfile an image, which is effectively like a copy of a hard drive which will be the basis of each container when it is run. After building, you can publish the image if you want to share it with others, or you can keep it in your local Docker cache.
In either case, once you have an image, you can start one or more containers using it. Starting a container is like starting a virtual machine. The container must run on some host on which it can use the CPU, memory, and network resources.
What is the host of your container? And how is your container starting if you are not using a docker run
command or similar?
I am running the image from paperspace gradient notebooks. I want to work with them because they allow me to use GPU, which I don't have in my local device, and its free. They have some pre-made images to start what they call notebooks. This is how it looks: (all that grey text in the bexes is sample text)
So, I made/publish my own image. In order to run my own image, with my custom notebook I fill the following details:
Then, after a couple of minutes, there is a web address that I can access with the code-server running. For example: https://n0ipdpto.clg07azjl.paperspacegradient.com/?token=ad434ce2f9a6b1a30ae7e2e9b9bb3d7d And it looks like:
Ahhhhh, ok, now everything makes a lot more sense. I am not familiar with paperspace and I did not understand that you were using them as a host.
As for configuring ports through paperspace, it looks like this menu is where you would set up the port as I have been describing. Then you would need to set "Data Science: Jupyter Server URI" to https://n0ipdpto.clg07azjl.paperspacegradient.com:8889/?token=JUPYTER_SERVER_TOKEN
. However, I don't know how to get the token from the Jupyter Server instance started by VS Code.
As an alternative to using VS Code's Jupyter server, I would recommend starting your own:
8880
. Set up the forward in paperspace as 8880→8880
or whatever.jupyter lab --port=8880
http://127.0.0.1:8880/lab?token=319f987ca2bf35046368734e61ba362baa081b0dcdddc17e
. https://n0ipdpto.clg07azjl.paperspacegradient.com:8880/lab?token=319f987ca2bf35046368734e61ba362baa081b0dcdddc17e
lab
from the URL to get https://n0ipdpto.clg07azjl.paperspacegradient.com:8880/?token=319f987ca2bf35046368734e61ba362baa081b0dcdddc17e
and enter this into the "Data Science: Jupyter Server URI" setting. If all goes well, that should work.(For your use case, you might be better off just running the Jupyter server on paperspace and connecting to it as a remote Jupyter server using a plain VS Code installation. But let's take things one step at a time, and try the previous suggestion first.)
This surely is the most advance thing I have ever done with a computer. I tried to run the port as you said and then running the jupyter lab with the specified port. I got the URL, tho the token is just ... but it seemed to work as I could open a new notebook? from that link. Then I put that in the settings file aaand.... nothing really changed. The kernels remain unusable.
Once again, thank you very much for all your help, but I think it is time to surrender and admit this problem is bigger than I. I guess I will have to wait until the guys at open-svx release the new version and hope that would solve the problem. Really appreciate your help man, your dedication makes this a better world.
Well it's a very good sign that you're able to connect to JupyterLab from your browser.
I think I made a mistake in my instructions. Before you give up, you might want to make one last attempt and replace the host by localhost
in the "Data Science: Jupyter Server URI" setting. In particular, the setting should look like https://localhost:8880/?token=319f987ca2bf35046368734e61ba362baa081b0dcdddc17e
(but with a different token).
I suppose we can reopen this if someone else has this problem, but from my perspective this is solved.
You are right. All that could be done was done, so I will wait for the open-vsx update. Thank you very much for your help, great to talk to you.
@maresb thanks for going the extra mile helping out here! ♥️ We super appreciate it.
And sorry for all the trouble/workarounds you have to do @phisanti - hoping to get the Open VSX update out in the next month or two!
I'm having issues with jupyter too:
code-server --version
: code-server-3.9.3-linux-amd64.tar.gzJupyter Notebook should be functional
Jupyter Notebook displays but no cell can run
Previously I was able to fix this by following https://github.com/cdr/code-server/issues/2341 but that doesn't help any more.
I'd love to help, but that seems to me like some sort of javascript problem. With that I'm clueless.
Discussion migrated from https://github.com/cdr/code-server/issues/2341#issuecomment-796868354.
Under certain situations when running the MS Python extension, the following error can arise:
This problem seems to have began for some users with code-server v3.9.
To view the Jupyter output tab, open the "Command Palette..." and type
>Output: Focus on Output View
. In the drop-down menu near the top right of the output window, select "Jupyter".I run into this bug with my particularly complicated setup, so I strongly suspect that others who see this problem will have a different manifestation.
I run code-server in a Docker container which is based on Ubuntu 20.04. Moreover, I don't launch code-server directly, but I run it through JupyterLab via this plugin. It sets up a Jupyter proxy for the VS Code connection.
My suspicion is that something changed how the MS Python extension creates and connects with Jupyter kernels.
Looking at the Jupyter output, I see:
It seems that MS Python is trying to connect to my existing JupyterLab server, but it doesn't know the correct token and is thus getting rejected.
I managed to fix it by adding the setting
"python.dataScience.jupyterServerURI": "http://localhost:8888/?token=THEVALUEOFMYTOKEN",
to mysettings.json
, or equivalently by changing the setting "Data Science: Jupyter Server URI" fromlocal
tohttp://localhost:8888/?token=THEVALUEOFMYTOKEN
.