Azure Blob storage SFTP usage - EOF error

[NORDCOMPCZ]

Hi,

I'm trying to use the module in combination with Azure Blob Storage (SFTP with private key). Unfotunately with all tested methods - LStat, Open, Create general connection lost error appears. With debugger attached I was able to track it down to this line where EOF error appears.

Am I missing some Azure specific setting?

[puellanivis]

Random disconnects like this are usually because unsupported packets or features… but it doesn’t sound like anything here is a result of a misbehavior of our code, it’s Azure closing out your connection. 🤔

[NORDCOMPCZ]

Hi @puellanivis, thanks a lot for your prompt reply. I also don't suppose this might be a bug. Seems to be more a configuration issue. Unfortunately for me, other clients let me connect without any problem. In case you don't have any clue feel free to close this question as the problem seems to be somewhere at my end.

[drakkan]

Hello,

please post the output of sftp -vvvvv <sftp hostname> so we can see the negotiated cipher, MAC, KEX etc. and understand if they are supported

[NORDCOMPCZ]

Hi @drakkan. See below

debug3: spawning "C:\\Windows\\System32\\OpenSSH\\ssh.exe" "-oForwardX11 no" "-oPermitLocalCommand no" "-oClearAllForwardings yes" -v -v -v -i .\\key "-oForwardAgent no" -l account.home-folder.username -s -- account.blob.core.windows.net sftp as subprocess
OpenSSH_for_Windows_8.6p1, LibreSSL 3.4.3
debug1: Reading configuration data C:\\Users\\user/.ssh/config
debug1: C:\\Users\\user/.ssh/config line 44: include /home/kodet/.magento-cloud/ssh/*.config matched no files
debug1: C:\\Users\\user/.ssh/config line 45: Applying options for *
debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> 'C:\\Users\\user/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> 'C:\\Users\\user/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug2: resolving "account.blob.core.windows.net" port 22
debug3: ssh_connect_direct: entering
debug1: Connecting to account.blob.core.windows.net [20.60.22.36] port 22.
debug1: Connection established.
debug3: Failed to open file:C:/path/to/key.pub error:2
debug1: identity file .\\key type -1
debug3: Failed to open file:C:/path/to/key-cert error:2
debug3: Failed to open file:C:/path/to/key-cert.pub error:2
debug3: failed to open file:C:/path/to/key-cert error:2
debug1: identity file .\\key-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.6
debug1: Remote protocol version 2.0, remote software version AzureSSH_1.0.0
debug1: compat_banner: no match: AzureSSH_1.0.0
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to account.blob.core.windows.net:22 as 'account.home-folder.username'
debug3: record_hostkey: found key type ECDSA in file C:\\Users\\user/.ssh/known_hosts:99
debug3: load_hostkeys_file: loaded 1 keys from account.blob.core.windows.net
debug3: Failed to open file:C:/Users/user/.ssh/known_hosts2 error:2
debug1: load_hostkeys: fopen C:\\Users\\user/.ssh/known_hosts2: No such file or directory
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts: No such file or directory
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp256
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group-exchange-sha256,ext-info-s
debug2: host key algorithms: rsa-sha2-256,rsa-sha2-512,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384
debug2: ciphers ctos: aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr
debug2: ciphers stoc: aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr
debug2: MACs ctos: hmac-sha2-256,hmac-sha2-512,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com
debug2: MACs stoc: hmac-sha2-256,hmac-sha2-512,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com
debug2: compression ctos: none
debug2: compression stoc: none
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: ecdh-sha2-nistp256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:Ce+h+7thT5tt75ypIkWZ6+JnmQMZEl1N7Tt3Ldalb64
debug3: record_hostkey: found key type ECDSA in file C:\\Users\\user/.ssh/known_hosts:99
debug3: load_hostkeys_file: loaded 1 keys from account.blob.core.windows.net
debug3: Failed to open file:C:/Users/user/.ssh/known_hosts2 error:2
debug1: load_hostkeys: fopen C:\\Users\\user/.ssh/known_hosts2: No such file or directory
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts: No such file or directory
debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'account.blob.core.windows.net' is known and matches the ECDSA host key.
debug1: Found key in C:\\Users\\user/.ssh/known_hosts:99
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug3: unable to connect to pipe \\\\.\\pipe\\openssh-ssh-agent, error: 2
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: Will attempt key: .\\key  explicit
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: .\\key
debug3: sign_and_send_pubkey: RSA SHA256:9kmmdQdm4V4sCE65mR0mrAoO0e2WxDnbPOdCtpfwObE
debug3: sign_and_send_pubkey: signing using ssh-rsa SHA256:9kmmdQdm4V4sCE65mR0mrAoO0e2WxDnbPOdCtpfwObE
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 52
debug1: Authentication succeeded (publickey).
Authenticated to account.blob.core.windows.net ([20.60.22.36]:22).
debug2: fd 4 setting O_NONBLOCK
debug2: fd 5 setting O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Entering interactive session.
debug1: pledge: filesystem full
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 3 setting TCP_NODELAY
debug2: client_session2_setup: id 0
debug1: Sending subsystem: sftp
debug2: channel 0: request subsystem confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 4294967295 rmax 262143
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: subsystem request accepted on channel 0
debug2: Remote version: 3
Connected to account.blob.core.windows.net.
debug3: Sent message fd 6 T:16 I:1
debug3: SSH_FXP_REALPATH . -> / size 0
sftp>
debug2: channel 0: read<=0 rfd 4 len 562640715776
debug2: channel 0: read failed
debug2: chan_shutdown_read: channel 0: (i0 o0 sock -1 wfd 4 efd 6 [write])
debug2: channel 0: input open -> drain
debug3: send packet: type 1
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 r1 i1/0 o0/0 e[write]/0 fd -1/5/6 sock -1 cc -1)

debug3: fd 0 is not O_NONBLOCK
Transferred: sent 2876, received 1192 bytes, in 3.3 seconds
Bytes per second: sent 881.3, received 365.3
debug1: Exit status -1

[drakkan]

For what I see it should work. I have no way to test myself sorry

[NORDCOMPCZ]

@drakkan I can create a testing private key for you if it helps

[drakkan]

@drakkan I can create a testing private key for you if it helps

I don't typically provide free support like this, but I'm curious to understand if Azure SFTP works as an SFTP backend for SFTPGo. Please send a test private key to my email account, you can find it in each SFTPGo commit. Thanks

[NORDCOMPCZ]

As mentioned above, problem was in my code (just for future generations - SFTP function ran in my case via a helper module; problem was that I was trying the connection with each action and it created new instance every time. After I redesigned that it works as expected).

A massive thanks to @drakkan for help.

[puellanivis]

OH… wow. Yeah. 🤔 So, were you trying to perform actions on a sftp.Client that was already closed? If so, this is the sort of thing we could work some better feedback/error messages into.