pachadotdev / analogsea

Digital Ocean R client
https://pacha.dev/analogsea/
Apache License 2.0
155 stars 24 forks source link

SSH problem in docklet_pull (solved, but not ideal) #132

Closed berkorbay closed 7 years ago

berkorbay commented 7 years ago

Hi. I try to automate docklet creation and destroying. Though, I ran into some problems. I created my SSH key with a passphrase. I had DO to recognize it (by copy-pasting public key).

First problem I encountered was on the CRAN version. (Just tried today) I had the following error.

Error in derive_pubkey(key) :RAW() can only be applied to a 'raw', not a 'list'

Then I installed devtools version. Everything is fine. I got the nicest warnings.

Waiting for authentication in browser...
Press Esc/Ctrl + C to abort
Authentication complete.
Using default ssh keys: berk1
NB: This costs $0.01488 / hour until you droplet_delete() it

When I tried docklet_pull("rocker/hadleyverse") command, I got the following error.

Warning: Permanently added '198.199.97.135' (ECDSA) to the list of known hosts.
Permission denied (publickey).
Error: ssh failed
ssh -o BatchMode=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/var/folders/bt/kwy9rzkn519gcsqpsxbl76l40000gn/T//Rtmp2JYSuz/hosts root@198.199.97.135 'docker pull rocker/hadleyverse'

I tried to run same command again and got the following error.

Error: ssh failed (publickey).
ssh -o BatchMode=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/var/folders/bt/kwy9rzkn519gcsqpsxbl76l40000gn/T//Rtmp2JYSuz/hosts root@198.199.97.135 'docker pull rocker/hadleyverse'

I checked keys() and everything is fine. (Altered the fingerprint in the example, just in case.)

$berk1
<key> berk1 (8371036)
  Fingerprint: 3f:3b:f2:14:6c:41:c3:4f:21:2n:c8:65:de:ec:f0:f9

Then I tried to reach to the server from the terminal with the following prompt and got these messages. I was also asked my passphrase.

berkorbay$ ssh -i .ssh/id_rsa2 root@198.199.97.135
Warning: Identity file .ssh/id_rsa2 not accessible: No such file or directory.
The authenticity of host '198.199.97.135 (198.199.97.135)' can't be established.
ECDSA key fingerprint is SHA256:lLqhCRycTFD9Hcb1n+jWyodAVsoWS6w4D+21ZXhWhLM.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '198.199.97.135' (ECDSA) to the list of known hosts.
Saving password to keychain failed
Identity added: /Users/berkorbay/.ssh/id_rsa ((null))
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-64-generic x86_64)

When I returned to R and ran docklet_pull("rocker/hadleyverse") it performed without a problem.

Yet, it is not an ideal solution. This way I cannot automate DO processes. Is there a way to make it more seamless?

sckott commented 7 years ago

thx for the issue @berkorbay vacaying right now, will have a look monday

sckott commented 7 years ago

right, the Error in derive_pubkey(key) error is fixed in the dev version here

Looking at the ssh errors

@trestletech curious if you have thoughts on this. Here's the commit that created the ssh options now used https://github.com/sckott/analogsea/commit/d5f2335c02103f08bd0425e577baef2dad216969 - we use a temporary hosts file

I'm guessing that once a user has ssh'ed successfully once, after that it's all good, but maybe it's that first time that is the pain point.

trestletech commented 7 years ago

Is the berk1 key that analogsea is aware of the same key that you ended up using to SSH in? I notice that you specified a key .ssh/id_rsa2 which wasn't found so it fell back to .ssh/id_rsa.

I'm not familiar with how analogsea identifies the SSH keys on the system or which ones it would attempt to use to connect.

sckott commented 7 years ago

@berkorbay ^^ see above question

thx @trestletech good catch

berkorbay commented 7 years ago

Thanks for the responses!

Ok, I see your point about id_rsa2 - id_rsa stuff. Even though I made that mistake, "Terminal" fixed it for me. Though, it is not related to the main problem. I will write my train of thought in items so you can tell me what I am doing wrong.

Some missing information I should add. I used the development version and I am using a Macbook Air. R version 3.3.3.

I will reinstall from github and try again. I also encountered some other glitches in my attempt to automate the process that I should report later (not sure if they are reproducible or specific to the session).

berkorbay commented 7 years ago

I updated R, reinstalled github version of analogsea. This time it worked quite nicely. Btw, thanks for making such an amazing package.

sckott commented 7 years ago

great, glad it works @berkorbay !

benjaminrobinson commented 3 years ago

Hi, @sckott et. al! I'm running into a flavor of this error and am wondering if there's something I should be doing differently. I get an Error: libssh failure at 'connect': Timeout connecting to error when I'm trying to use docklet_pull after I've successfully created a droplet using docklet_create. If I try to run docklet_pull again, I get an New server key:...Error: Authentication with ssh server failed message, which was how I found this GH issue.

And here's my reprex/sessioninfo:

library(analogsea)
#> Warning: package 'analogsea' was built under R version 3.6.3
#> 
#> Attaching package: 'analogsea'
#> The following object is masked from 'package:graphics':
#> 
#>     image
keys()
#> $`BJR Personal Laptop`
#> <key> BJR Personal Laptop (CENSORED for reprex)
#>   Fingerprint: CENSORED for reprex
droplet1 <- docklet_create(region = "nyc1", size = "s-4vcpu-8gb")
#> Using default ssh keys: BJR Personal Laptop
#> NB: This costs $0.05952 / hour until you droplet_delete() it
#> Waiting for create ..................

droplet(droplet1$id) %>%
  docklet_pull("benjaminrobinson/gtm-docker")
#> Error: libssh failure at 'connect': Timeout connecting to CENSORED for reprex

Created on 2021-01-03 by the reprex package (v0.3.0)

Session info ``` r devtools::session_info() #> - Session info --------------------------------------------------------------- #> setting value #> version R version 3.6.1 (2019-07-05) #> os Windows 10 x64 #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate English_United States.1252 #> ctype English_United States.1252 #> tz America/New_York #> date 2021-01-03 #> #> - Packages ------------------------------------------------------------------- #> package * version date lib source #> analogsea * 0.9.0 2020-10-08 [1] CRAN (R 3.6.3) #> askpass 1.1 2019-01-13 [1] CRAN (R 3.6.1) #> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.1) #> backports 1.1.5 2019-10-02 [1] CRAN (R 3.6.1) #> callr 3.3.2 2019-09-22 [1] CRAN (R 3.6.1) #> cli 2.0.2 2020-02-28 [1] CRAN (R 3.6.3) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.1) #> credentials 1.3.0 2020-07-21 [1] CRAN (R 3.6.3) #> curl 4.3 2019-12-02 [1] CRAN (R 3.6.1) #> desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.2) #> devtools 2.2.1 2019-09-24 [1] CRAN (R 3.6.2) #> digest 0.6.23 2019-11-23 [1] CRAN (R 3.6.1) #> ellipsis 0.3.1 2020-05-15 [1] CRAN (R 3.6.3) #> evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.1) #> fansi 0.4.0 2018-10-05 [1] CRAN (R 3.6.1) #> fs 1.5.0 2020-07-31 [1] CRAN (R 3.6.3) #> glue 1.4.1 2020-05-13 [1] CRAN (R 3.6.3) #> highr 0.8 2019-03-20 [1] CRAN (R 3.6.1) #> htmltools 0.5.0 2020-06-16 [1] CRAN (R 3.6.3) #> httr 1.4.1 2019-08-05 [1] CRAN (R 3.6.1) #> jsonlite 1.6.1 2020-02-02 [1] CRAN (R 3.6.3) #> knitr 1.28 2020-02-06 [1] CRAN (R 3.6.2) #> magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.1) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.1) #> openssl 1.4.1 2019-07-18 [1] CRAN (R 3.6.1) #> pkgbuild 1.0.6 2019-10-09 [1] CRAN (R 3.6.2) #> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.2) #> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.6.1) #> processx 3.4.1 2019-07-18 [1] CRAN (R 3.6.1) #> ps 1.3.0 2018-12-21 [1] CRAN (R 3.6.1) #> R6 2.4.1 2019-11-12 [1] CRAN (R 3.6.1) #> remotes 2.1.0 2019-06-24 [1] CRAN (R 3.6.2) #> rlang 0.4.7 2020-07-09 [1] CRAN (R 3.6.3) #> rmarkdown 1.18 2019-11-27 [1] CRAN (R 3.6.1) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.1) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.2) #> ssh 0.7.0 2020-07-21 [1] CRAN (R 3.6.3) #> stringi 1.4.3 2019-03-12 [1] CRAN (R 3.6.0) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.1) #> sys 3.3 2019-08-21 [1] CRAN (R 3.6.1) #> testthat 2.3.1 2019-12-01 [1] CRAN (R 3.6.2) #> usethis 1.6.3 2020-09-17 [1] CRAN (R 3.6.3) #> withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.1) #> xfun 0.11 2019-11-12 [1] CRAN (R 3.6.1) #> yaml 2.2.1 2020-02-01 [1] CRAN (R 3.6.2) #> #> [1] C:/Users/benjr/Documents/R/win-library/3.6 #> [2] C:/Program Files/R/R-3.6.1/library ```
sckott commented 3 years ago

Thanks for the issue @benjaminrobinson

First thing I notice is that benjaminrobinson/gtm-docker is invalid somehow. I can't pull it to my local machine or on a DO server. Any ideas?

benjaminrobinson commented 3 years ago

Hi, @sckott! Thank you so much for the package and the quick reply. This is my first time using Docker and I've been trying to make and use my own image. If I've done so unsuccessfully, please let me know! Here's the link to the repository I have created and published on the Docker Hub. I've noticed that there isn't a "Dockerfile" that shows up but I do have some "tags" listed there. Is there something else I need to do to get it set up so you can pull it potentially? Thanks again!

https://hub.docker.com/r/benjaminrobinson/gtm-docker

sckott commented 3 years ago

I don't have bandwidth to help with docker images - let's just use a different docker image from dockerhub - one you know that works - so we're not confounding the issue of the docker image and any issues in this package. - can you share another reprex but with an image we know that works

benjaminrobinson commented 3 years ago

Hi, @sckott! I used the "rocker/tidyverse" image from dockerhub and got the same error message.

library(analogsea)
#> Warning: package 'analogsea' was built under R version 3.6.3
#> 
#> Attaching package: 'analogsea'
#> The following object is masked from 'package:graphics':
#> 
#>     image
keys()
#> $`BJR Personal Laptop`
#> <key> BJR Personal Laptop (XXXXXXXX)
#>   Fingerprint: XXXXXXXXXXXXXXXXXXXXXXXXXXXXX

droplet1 <- docklet_create(region = "nyc1", size = "s-4vcpu-8gb")
#> Using default ssh keys: BJR Personal Laptop
#> NB: This costs $0.05952 / hour until you droplet_delete() it
#> Waiting for create .............................

droplet(droplet1$id) %>%
  docklet_pull("rocker/tidyverse")
#> Error: libssh failure at 'connect': Timeout connecting to XX.XXX.X.XXX

Created on 2021-01-08 by the reprex package (v0.3.0)

sckott commented 3 years ago

Thanks. That still works for me.

Can you connect to the server from your command line via ssh successfully?

In R, does a very simple thing work? Try:

droplet_ssh(d, "whoami")
benjaminrobinson commented 3 years ago

Here's what I got when running the code you just shared: droplet1<- docklet_create() droplet1 <- droplet(droplet1$id) droplet_ssh(droplet1, "whoami")

This yielded the following result: New server key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Error: Authentication with ssh server failed

sckott commented 3 years ago

Thanks for that.

What i meant by "Can you connect to the server from your command line via ssh successfully?" was can you use your command line/terminal/shell outside of R to connect to the server. That will help us see if your ssh key setup is working correctly. e.g., in your shell

ssh -i .ssh/id_rsa username@your-server-ip-address
benjaminrobinson commented 3 years ago

Hi, @sckott! Sorry for not replying sooner. I was able to connect to the Droplet in the terminal! It asked me for my password and then I realized that I could apply some of the same logic to the docklet_pull I was trying to do before!

droplet(droplet1$id) %>% docklet_pull("rocker/tidyverse", ssh_passwd = 'PASSWORD', keyfile = 'LOCATION')

This worked for me! Thanks for being patient with me and helping me understand my problem! I'm guessing you don't have much time to help with my docker image but if you know of any resources that you could point me to that would be great! Thanks again!

sckott commented 3 years ago

Great, glad it worked.

I can't help with images, but perhaps these might help:

Rocker questions on stackoverflow https://stackoverflow.com/search?q=%5Br%5D+rocker

Docker community info https://www.docker.com/docker-community