search

ropensci / ssh

Native SSH client in R based on libssh
https://docs.ropensci.org/ssh
Other
127 stars 21 forks source link

SCP errors on Ubuntu after openssh-server patch release #20

Closed mtvincen closed 5 years ago

mtvincen commented 5 years ago

I have been using your package for a while but recently I have not been unable to upload files from a windows computer to an Ubuntu computer. The code I had written worked last week, but since then I think there may have been some updates to Ubuntu. I am still able to use the ssh_exec_wait and scp_download. I attempted to use the scp_upload function sending the file to a Mac and it seems to work okay. Any help with fixing this would be much appreciated. Below is the error code and traceback

scp_upload(session,files=c('Start.tar.gz','condor.sub','mfcl.X86_64.LINUX.bat'),to="SKJ/GrowthStartingPoint/SamePhaseEst") Error: Failed to create: SKJ/GrowthStartingPoint/SamePhaseEst (no such directory?) traceback() 1: scp_upload(session, files = c("Start.tar.gz", "condor.sub", "mfcl.X86_64.LINUX.bat"), to = "SKJ/GrowthStartingPoint/SamePhaseEst")

jjesusfilho commented 5 years ago

I am having the same problem. Until last week I was transferring files to Ubuntu from my Mac, but this week I am getting the same error (no such directory?).

mscbuck commented 5 years ago

Would just like to add that I also suddenly have run into this error. Uploading to an Ubuntu droplet on Digital Ocean suddenly fails with a "no such directory" error? I have tried all options as well when it comes to specifying directories. I have confirmed that the directory below exists. I have run a manual SCP command and it works, but through R it does not:

I'm on Mac here, uploading to a Ubuntu 16.04.5 machine

ssh_scp_init: Initializing scp session write recursive on location 'var/plumber/testapp' channel_open: Creating a channel 45 with 64000 window and 32768 max packet ssh_packet_channel_open_conf: Received a CHANNEL_OPEN_CONFIRMATION for channel 45:2 ssh_packet_channel_open_conf: Remote window : 0, maxpacket : 32768 channel_rcv_change_window: Adding 2097152 bytes to channel (45:2) (from 0 bytes) channel_request: Channel request exec success grow_window: growing window (channel 45:2) to 1280000 bytes ssh_scp_push_directory: scp status code 1d not valid Error: Failed to create: var/plumber/testapp (no such directory?)

jeroen commented 5 years ago

OK here is a changelog from Ubuntu that mentions SCP security patches: http://changelogs.ubuntu.com/changelogs/pool/main/o/openssh/openssh_7.2p2-4ubuntu2.7/changelog

mscbuck commented 5 years ago

Connecting to Ubuntu 16.04.5 x64. This part actually had me interested, because I don't believe I had this issue with 16.04.4, but I cannot find anything in the changelogs that might be related to this issue **Edit: Looks like you found something

Here is my output from "sshd -v" (which errors, but will tell you the version:

OpenSSH_7.2p2 Ubuntu-4ubuntu2.7, OpenSSL 1.0.2g 1 Mar 2016

Here is my SessionInfo():

`R version 3.5.1 (2018-07-02) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS 10.14.3

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] analogsea_0.6.5.9110 plumber_0.4.7

loaded via a namespace (and not attached): [1] Rcpp_1.0.0 ssh_0.3 digest_0.6.18 packrat_0.5.0 crayon_1.3.4 later_0.7.3 aws.signature_0.4.4 [8] R6_2.3.0 jsonlite_1.6 magrittr_1.5 httr_1.4.0 stringi_1.2.4 curl_3.3 rstudioapi_0.8
[15] promises_1.0.1 xml2_1.2.0 tools_3.5.1 aws.s3_0.3.12 yaml_2.2.0 httpuv_1.4.5 compiler_3.5.1
[22] base64enc_0.1-3 openssl_1.1 `

mtvincen commented 5 years ago

sessionInfo() R version 3.5.1 (2018-07-02) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 17134)

Matrix products: default

locale: [1] LC_COLLATE=English_World.1252 LC_CTYPE=English_World.1252
[3] LC_MONETARY=English_World.1252 LC_NUMERIC=C
[5] LC_TIME=English_World.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] ssh_0.3 colorRamps_2.3 ggplot2_3.1.0
[4] R4MFCL_0.4.1.2017.12.18 data.table_1.11.8 dplyr_0.7.8
[7] FLR4MFCL_1.1.1 FLCore_2.6.10.9001 lattice_0.20-38

loaded via a namespace (and not attached): [1] tidyselect_0.2.5 purrr_0.2.5 reshape2_1.4.3 tcltk_3.5.1
[5] colorspace_1.3-2 viridisLite_0.3.0 stats4_3.5.1 blob_1.1.1
[9] rlang_0.3.0.1 pillar_1.3.0 withr_2.1.2 glue_1.3.0
[13] DBI_1.0.0 sp_1.3-1 bit64_0.9-7 bindrcpp_0.2.2
[17] jpeg_0.1-8 bindr_0.1.1 plyr_1.8.4 stringr_1.4.0
[21] munsell_0.5.0 gtable_0.2.0 RgoogleMaps_1.4.3 mapproj_1.2.6
[25] memoise_1.1.0 proto_1.0.0 Rcpp_1.0.0 geosphere_1.5-7
[29] xtable_1.8-3 scales_1.0.0 bit_1.1-14 mapdata_2.3.0
[33] gridExtra_2.3 rjson_0.2.20 png_0.1-7 digest_0.6.18
[37] stringi_1.2.4 getPass_0.2-2 grid_3.5.1 tools_3.5.1
[41] magrittr_1.5 maps_3.3.0 lazyeval_0.2.1 tibble_1.4.2
[45] RSQLite_2.1.1 crayon_1.3.4 tidyr_0.8.2 pkgconfig_2.0.2
[49] MASS_7.3-51.1 RODBC_1.3-15 Matrix_1.2-15 assertthat_0.2.0 [53] viridis_0.5.1 R6_2.3.0 ggmap_2.6.1 compiler_3.5.1

session

connected: matthewv@nouofpcalc02:22 server: d5:b5:f1:43:cd:9a:9a:2d:27:ab:d6:e6:e0:11:6a:e4:80:65:75:b4 Connected to : Ubuntu 16.04.5 LTS NOUOFPCALC02 ssh-pty
jeroen commented 5 years ago

Ubuntu @mdeslaur pushed a new version of ssh-server on January 31 2019 with several SCP security patches. I think these patches have somehow broken the R client. I'll try to figure out what the issue is and how to work around it.

openssh (1:7.2p2-4ubuntu2.7) xenial-security; urgency=medium

  * SECURITY UPDATE: access restrictions bypass in scp
    - debian/patches/CVE-2018-20685.patch: disallow empty filenames
      or ones that refer to the current directory in scp.c.
    - CVE-2018-20685
  * SECURITY UPDATE: scp client spoofing via object name
    - debian/patches/CVE-2019-6109.patch: make sure the filenames match
      the wildcard specified by the user, and add new flag to relax the new
      restrictions in scp.c, scp.1.
    - CVE-2019-6109
  * SECURITY UPDATE: scp client missing received object name validation
    - debian/patches/CVE-2019-6111-pre1.patch: backport snmprintf from
      newer OpenSSH in Makefile.in, utf8.c, utf8.h, configure.ac.
    - debian/patches/CVE-2019-6111-pre2.patch: update vis.h and vis.c from
      newer OpenSSH.
    - debian/patches/CVE-2019-6111-1.patch: sanitize scp filenames via
      snmprintf in atomicio.c, progressmeter.c, progressmeter.h,
      scp.c, sftp-client.c.
    - debian/patches/CVE-2019-6111-2.patch: force progressmeter updates in
      progressmeter.c, progressmeter.h, scp.c, sftp-client.c.
    - CVE-2019-6111

 -- Marc Deslauriers <marc.deslauriers@ubuntu.com>  Thu, 31 Jan 2019 09:03:12 -0500
jeroen commented 5 years ago

Could somebody test if other versions of Debian/Ubuntu/Fedora are affected, or if this bug only appears for Ubuntu 16.04?

mtvincen commented 5 years ago

Sorry don't have access to any other versions

mscbuck commented 5 years ago

Could somebody test if other versions of Debian/Ubuntu/Fedora are affected, or if this bug only appears for Ubuntu 16.04?

Will test with an Ubuntu 18.04 droplet on DO and report back...

**EDIT: It worked fine with Ubuntu 18.04!

jeroen commented 5 years ago

**EDIT: It worked fine with Ubuntu 18.04!

OK so the Ubuntu folks likely broke something when they backported security patches to Xenial. I'll see if there is a way to work around this.

mdeslaur commented 5 years ago

Please file a bug here once you figure out what exact behaviour changed in Xenial, and I'll fix it:

https://bugs.launchpad.net/ubuntu/+source/openssh/+filebug

Thanks!

jeroen commented 5 years ago

I tried uploading a file to a Xenial server before and after updating to the latest ssh-server. Before:

> scp_upload(test, "DESCRIPTION", "~/")
ssh_scp_init: Initializing scp session write recursive on location '~/'
channel_open: Creating a channel 43 with 64000 window and 32768 max packet
ssh_packet_global_request: Received SSH_MSG_GLOBAL_REQUEST packet
ssh_packet_global_request: UNKNOWN SSH_MSG_GLOBAL_REQUEST hostkeys-00@openssh.com 0
ssh_packet_process: Couldn't do anything with packet type 80
ssh_packet_channel_open_conf: Received a CHANNEL_OPEN_CONFIRMATION for channel 43:0
ssh_packet_channel_open_conf: Remote window : 0, maxpacket : 32768
channel_rcv_change_window: Adding 2097152 bytes to channel (43:0) (from 0 bytes)
channel_request: Channel request exec success
grow_window: growing window (channel 43:0) to 1280000 bytes
ssh_scp_push_file64: SCP pushing file DESCRIPTION, size 723 with permissions '0644'

After:


> scp_upload(test, "DESCRIPTION", "~/")
ssh_scp_init: Initializing scp session write recursive on location '~/'
channel_open: Creating a channel 43 with 64000 window and 32768 max packet
ssh_packet_global_request: Received SSH_MSG_GLOBAL_REQUEST packet
ssh_packet_global_request: UNKNOWN SSH_MSG_GLOBAL_REQUEST hostkeys-00@openssh.com 0
ssh_packet_process: Couldn't do anything with packet type 80
ssh_packet_channel_open_conf: Received a CHANNEL_OPEN_CONFIRMATION for channel 43:0
ssh_packet_channel_open_conf: Remote window : 0, maxpacket : 32768
channel_rcv_change_window: Adding 2097152 bytes to channel (43:0) (from 0 bytes)
channel_request: Channel request exec success
grow_window: growing window (channel 43:0) to 1280000 bytes
ssh_scp_push_directory: scp status code 1d not valid
Error: Failed to create: ~/ (no such directory?)
jeroen commented 5 years ago

The problem appears here: https://github.com/ropensci/ssh/blob/626e77662e459b83309dbaff69ffbaa52c878fb7/src/scp.c#L137-L139

The first line starts the scp session in the given target directory on the server, which is still ok. But then we call ssh_scp_push_directory(scp, ".", 493L) which is supposed to create the target directory in case it does not exist yet. This call now seems to return an error code, before it would succeed.

jeroen commented 5 years ago

I've pushed a fix. Please test:

remotes::install_github("ropensci/ssh")
mtvincen commented 5 years ago

The new version appears to work for uploading files. Thank you

However, I noticed some strange functionality when creating the session. The login terminal now has a different appearance and the username says NA instead of my user name. Upon successfully logging in I received the message: "Found known server key: d5:b5:f1:43:cd :9a:9a:2d:27:ab :d6:e6:e0:11:6a:e4:80:65:75:b4 " I don't know if this will affect other people when logging in or not if they don't already have a key set up.

jeroen commented 5 years ago

What do you mean with the terminal has a different appearance and the username says na? Can you share example code or screenshot?

mtvincen commented 5 years ago

image

jeroen commented 5 years ago

@mtvincen OK I've made the password prompt a bit better on Windows now.

jeroen commented 5 years ago

Assuming this issue is fixed now, and submitted ssh 0.4 to CRAN. If anyone notices additional problems, please open a new issue.

jeroen commented 5 years ago

This is all on CRAN now, please update:

install.packages("ssh")