demotomohiro / Internet-of-Tomohiro

My blog site
MIT License
2 stars 0 forks source link

google_colab/ssh.en #1

Open utterances-bot opened 4 years ago

utterances-bot commented 4 years ago

How to access Google Colaboratory using ssh

Explains how to run ssh server on Google Colaboratory and use it via ssh

https://internet-of-tomohiro.netlify.com/google_colab/ssh.en.html

rogelioamancisidor commented 4 years ago

After i have stablished the ssh connection with colab, can i see my files in gdrive from the terminal? because I dont manage to see anything

demotomohiro commented 4 years ago

Copy and paste following code to a cell and run.

from google.colab import drive
drive.mount('/gdrive')

Your google drive will be mount on /gdrive and you can see it from the terminal. You need to be root to access that directory.

I found that code on code snippets "Mounting Google Drive in your VM". That is displayed when I press ctrl+alt+p keys on Google colab.

rogelioamancisidor commented 4 years ago

do you mean run <sudo ls /gdrive> from terminal? I have tried this and use the root password generated after running your script, but the password seems to be wrong. The gdrive is mounted and I can access to it from colab webapage.

rogelioamancisidor commented 4 years ago

I also tried to loggin as <ssh root@0.tcp.eu.ngrok.io> using the root pasword generated but it is not accepted

demotomohiro commented 4 years ago

sudo command in Google colab seems not configured properly. You can become root user with su command. Just run su command and copy & paste the root password.

remocolab uses default config of openssh-server. You cannot login as root using password in default.

rogelioamancisidor commented 4 years ago

Thanks a lot! <su> command works. I have some further questions ...

demotomohiro commented 4 years ago

is it possible to automate the authoken step from ngrok, the token is always the same isnt?

You can do that by replacing

remocolab.setupSSHD()

to

remocolab._setupSSHDImpl("Your authtoken", "eu")

But I don't think it is a good idea because authtoken seems like something should be secret. If you forget that there is your authtoken in the notebook and publish it, anyone can use your authtoken. That why setupSSHD function doesn't take authtoken but ask it everytime. I don't know whether google colab's vm can get authtoken automatically and securely.

what about the region? assuming I always want to use eu region for example

Pass your region to remocolab.setupSSHD or remocolab.setupVNC like:

remocolab.setupSSHD("eu")

or

remocolab.setupVNC("eu")

is it possible to use container images with colab?

If it means about docker, I tried it on colab before. But it seems colab is already working on docker and running docker on docker is very hard. I gave up.

libinruan commented 4 years ago

Hello, I can successfully see the virtual desktop following your guide. remocolab works like a charm. I got this message below when I tried to install PyCharm in the remote server. Is there any resolution to this issue?

Start Failed: Failed to initialize graphics environment
java.awt.AWTError: Can't connect to X11 window server using ':1.0' as the value of the DISPLAY variable.
libinruan commented 4 years ago

I found the solution to my problem stated right above: when making ssh access to the colab server with the regular user account, continue use the regular user account to install Pycharm. Otherwise, the installation won't success.

libinruan commented 4 years ago

Is it possible to reboot the Colab server and SSH to the same machine later?

In my case, I want to downgrade CUDA from version 10.1 to 10.0 so that I can run a specific version of Tensorflow (2.0.0). A reboot is required but I don't know how to reboot and then SSH back to the same machine.

demotomohiro commented 4 years ago

There is already cuda-10.0 in colab machine without installing it.

~$ /usr/local/cuda-10.0/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130

It seems colab instance is working on docker or something like docker. I don't think you cannot reboot the server. I think installing old cuda requires reboot because that try to install specific version of kernel module. But as far as I know, you cannot delete or update kernel module used on colab.

libinruan commented 4 years ago

Thanks for the detailed information! It saves me a lot of trouble investigating this issue.

On Sun, Jul 26, 2020 at 10:30 AM Tomohiro notifications@github.com wrote:

There is already cuda-10.0 in colab machine without installing it.

~$ /usr/local/cuda-10.0/bin/nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2018 NVIDIA Corporation Built on Sat_Aug_25_21:08:01_CDT_2018 Cuda compilation tools, release 10.0, V10.0.130

It seems colab instance is working on docker or something like docker. I don't think you cannot reboot the server. I think installing old cuda requires reboot because that try to install specific version of kernel module. But as far as I know, you cannot delete or update kernel module used on colab.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/demotomohiro/Internet-of-Tomohiro/issues/1#issuecomment-663995051, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGXOK6IAL5LHXSSMRXXF2HTR5Q4W3ANCNFSM4JRYCSWA .