liximomo / vscode-sftp

Super fast sftp/ftp extension for VS Code
MIT License
1.5k stars 259 forks source link

Unable to upload on safe using SFTP to remote academic computing cluster #796

Open brando90 opened 4 years ago

brando90 commented 4 years ago

Do you read the FAQ?

Describe the bug When I do file on save the file in the server does not change but the logs don't show any obvious error.

To Reproduce I don't know how to reproduce. SFTP was working fine (and there doesn't seem to be a change in the SFTP codebase recently).

Expected behavior I should be able to see the changes I made locally on the server

Screenshots N\A

Desktop (please complete the following information):

Version: 1.46.1 Commit: cd9ea6488829f560dc949a8b2fb789f3cdc05f5d Date: 2020-06-17T21:17:14.222Z (2 wks ago) Electron: 7.3.1 Chrome: 78.0.3904.130 Node.js: 12.8.1 V8: 7.8.279.23-electron.0 OS: Darwin x64 19.5.0

Extension Logs from Startup - required

[07-08 12:12:27] [debug] register command "Cancel All Transfer" from "./commandCancelAllTransfer.ts"
[07-08 12:12:27] [debug] register command "Config" from "./commandConfig.ts"
[07-08 12:12:27] [debug] register command "List Active Folder" from "./commandListActiveFolder.ts"
[07-08 12:12:27] [debug] register command "Open Ssh Connection" from "./commandOpenSshConnection.ts"
[07-08 12:12:27] [debug] register command "Set Profile" from "./commandSetProfile.ts"
[07-08 12:12:27] [debug] register command "Toggle Output Panel" from "./commandToggleOutputPanel.ts"
[07-08 12:12:27] [debug] register command "Upload Changed Files" from "./commandUploadChangedFiles.ts"
[07-08 12:12:27] [debug] register command "Delete Remote" from "./fileCommandDeleteRemote.ts"
[07-08 12:12:27] [debug] register command "Diff" from "./fileCommandDiff.ts"
[07-08 12:12:27] [debug] register command "Diff Active File" from "./fileCommandDiffActiveFile.ts"
[07-08 12:12:27] [debug] register command "Download" from "./fileCommandDownload.ts"
[07-08 12:12:27] [debug] register command "Download Active File" from "./fileCommandDownloadActiveFile.ts"
[07-08 12:12:27] [debug] register command "Download Active Folder" from "./fileCommandDownloadActiveFolder.ts"
[07-08 12:12:27] [debug] register command "Download File" from "./fileCommandDownloadFile.ts"
[07-08 12:12:27] [debug] register command "Download Folder" from "./fileCommandDownloadFolder.ts"
[07-08 12:12:27] [debug] register command "Download Force" from "./fileCommandDownloadForce.ts"
[07-08 12:12:27] [debug] register command "Download Project" from "./fileCommandDownloadProject.ts"
[07-08 12:12:27] [debug] register command "Edit In Local" from "./fileCommandEditInLocal.ts"
[07-08 12:12:27] [debug] register command "List" from "./fileCommandList.ts"
[07-08 12:12:27] [debug] register command "List All" from "./fileCommandListAll.ts"
[07-08 12:12:27] [debug] register command "Reveal In Explorer" from "./fileCommandRevealInExplorer.ts"
[07-08 12:12:27] [debug] register command "Reveal In Remote Explorer" from "./fileCommandRevealInRemoteExplorer.ts"
[07-08 12:12:27] [debug] register command "Sync Both Directions" from "./fileCommandSyncBothDirections.ts"
[07-08 12:12:27] [debug] register command "Sync Local To Remote" from "./fileCommandSyncLocalToRemote.ts"
[07-08 12:12:27] [debug] register command "Sync Remote To Local" from "./fileCommandSyncRemoteToLocal.ts"
[07-08 12:12:27] [debug] register command "Upload" from "./fileCommandUpload.ts"
[07-08 12:12:27] [debug] register command "Upload Active File" from "./fileCommandUploadActiveFile.ts"
[07-08 12:12:27] [debug] register command "Upload Active Folder" from "./fileCommandUploadActiveFolder.ts"
[07-08 12:12:27] [debug] register command "Upload File" from "./fileCommandUploadFile.ts"
[07-08 12:12:27] [debug] register command "Upload Folder" from "./fileCommandUploadFolder.ts"
[07-08 12:12:27] [debug] register command "Upload Force" from "./fileCommandUploadForce.ts"
[07-08 12:12:27] [debug] register command "Upload Project" from "./fileCommandUploadProject.ts"
[07-08 12:12:27] [info] config at /Users/brando {"remotePath":"./","uploadOnSave":false,"downloadOnOpen":false,"ignore":[],"concurrency":4,"protocol":"sftp","connectTimeout":10000,"interactiveAuth":false,"secure":false,"remoteTimeOffsetInHours":0,"name":"automl-meta-learning","profiles":{"prod":{"name":"Vision-cluster","host":"vision-sched.cs.illinois.edu","port":22,"username":"miranda9","password":"***","remotePath":"/home/miranda9/automl-meta-learning","uploadOnSave":true,"downloadOnOpen":false},"dev":{"name":"HAL-NCSA-cluster","host":"hal.ncsa.illinois.edu","port":22,"username":"miranda9","password":"************","remotePath":"/home/miranda9/automl-meta-learning","uploadOnSave":true,"downloadOnOpen":false}},"defaultProfile":"prod"}

when I do save then this is the sftp output:

[07-08 12:14:01] [info] Using profile: prod
[07-08 12:14:10] [info] Using profile: prod
[07-08 12:14:10] [info] [file-save] /Users/brando/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py
[07-08 12:14:10] [info] Using profile: prod
[07-08 12:14:10] [trace] handle upload file for /Users/brando/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py
[07-08 12:14:10] [debug] Local ident: 'SSH-2.0-ssh2js0.4.5'
[07-08 12:14:10] [debug] Client: Trying vision-sched.cs.illinois.edu on port 22 ...
[07-08 12:14:10] [debug] Client: Connected
[07-08 12:14:10] [debug] Remote ident: 'SSH-2.0-OpenSSH_7.4'
[07-08 12:14:10] [debug] Outgoing: Writing KEXINIT
[07-08 12:14:10] [debug] DEBUG: Comparing KEXINITs ...
[07-08 12:14:10] [debug] (local) KEX algorithms: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1
[07-08 12:14:10] [debug] (remote) KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
[07-08 12:14:10] [debug] KEX algorithm: ecdh-sha2-nistp256
[07-08 12:14:10] [debug] (local) Host key formats: ssh-ed25519,ssh-rsa,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
[07-08 12:14:10] [debug] (remote) Host key formats: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
[07-08 12:14:10] [debug] Host key format: ssh-ed25519
[07-08 12:14:10] [debug] (local) Client->Server ciphers: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes128-gcm@openssh.com,aes256-gcm,aes256-gcm@openssh.com
[07-08 12:14:10] [debug] (remote) Client->Server ciphers: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
[07-08 12:14:10] [debug] Client->Server Cipher: aes128-ctr
[07-08 12:14:10] [debug] (local) Server->Client ciphers: aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm,aes128-gcm@openssh.com,aes256-gcm,aes256-gcm@openssh.com
[07-08 12:14:10] [debug] (remote) Server->Client ciphers: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
[07-08 12:14:10] [debug] Server->Client Cipher: aes128-ctr
[07-08 12:14:10] [debug] (local) Client->Server HMAC algorithms: hmac-sha2-256,hmac-sha2-512,hmac-sha1
[07-08 12:14:10] [debug] (remote) Client->Server HMAC algorithms: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
[07-08 12:14:10] [debug] Client->Server HMAC algorithm: hmac-sha2-256
[07-08 12:14:10] [debug] (local) Server->Client HMAC algorithms: hmac-sha2-256,hmac-sha2-512,hmac-sha1
[07-08 12:14:10] [debug] (remote) Server->Client HMAC algorithms: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
[07-08 12:14:10] [debug] Server->Client HMAC algorithm: hmac-sha2-256
[07-08 12:14:10] [debug] (local) Client->Server compression algorithms: none,zlib@openssh.com,zlib
[07-08 12:14:10] [debug] (remote) Client->Server compression algorithms: none,zlib@openssh.com
[07-08 12:14:10] [debug] Client->Server compression algorithm: none
[07-08 12:14:10] [debug] (local) Server->Client compression algorithms: none,zlib@openssh.com,zlib
[07-08 12:14:10] [debug] (remote) Server->Client compression algorithms: none,zlib@openssh.com
[07-08 12:14:10] [debug] Server->Client compression algorithm: none
[07-08 12:14:10] [debug] Outgoing: Writing KEXECDH_INIT
[07-08 12:14:11] [debug] DEBUG: Checking host key format
[07-08 12:14:11] [debug] DEBUG: Checking signature format
[07-08 12:14:11] [debug] DEBUG: Verifying host fingerprint
[07-08 12:14:11] [debug] DEBUG: Host accepted by default (no verification)
[07-08 12:14:11] [debug] DEBUG: Verifying signature
[07-08 12:14:11] [debug] Outgoing: Writing NEWKEYS
[07-08 12:14:11] [debug] Outgoing: Writing SERVICE_REQUEST (ssh-userauth)
[07-08 12:14:11] [debug] Outgoing: Writing USERAUTH_REQUEST (none)
[07-08 12:14:11] [debug] Client: none auth failed
[07-08 12:14:11] [debug] Outgoing: Writing USERAUTH_REQUEST (password)
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_OPEN (0, session)
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_REQUEST (0, subsystem: sftp)
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:11] [debug] Outgoing: Writing MKDIR
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:11] [debug] Outgoing: Writing LSTAT
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:11] [debug] Outgoing: Writing OPEN
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:11] [debug] Outgoing: Writing FSTAT
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:11] [debug] Outgoing: Writing FSETSTAT
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:11] [debug] Outgoing: Writing WRITE
[07-08 12:14:11] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:12] [debug] Outgoing: Writing FSETSTAT
[07-08 12:14:12] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:12] [debug] Outgoing: Writing CLOSE
[07-08 12:14:12] [debug] Outgoing: Writing CHANNEL_DATA (0)
[07-08 12:14:12] [info] local ➞ remote /Users/brando/automl-meta-learning/automl-proj/experiments_meta_learning/meta_learning_experiments_submission.py

the most notable output:

[07-08 12:14:11] [debug] Client: none auth failed
brando90 commented 4 years ago

note I did change the ulimit of my mac:

(automl-meta-learning) brando~ ❯ echo kern.maxfiles=65536 | sudo tee -a /etc/sysctl.conf
echo kern.maxfilesperproc=65536 | sudo tee -a /etc/sysctl.conf
sudo sysctl -w kern.maxfiles=65536
sudo sysctl -w kern.maxfilesperproc=65536
ulimit -n 65536
Password:
kern.maxfiles=65536
kern.maxfilesperproc=65536
kern.maxfiles: 65536 -> 65536
kern.maxfilesperproc: 65536 -> 65536
brando90 commented 4 years ago

I also tried the suggestions https://github.com/liximomo/vscode-sftp/issues/499 and they failed.

Did the following but it failed:

having those lines in sftp.json should work (and work for me):

"protocol": "ftp",
"secure": true
If you're having certificate problems, include (and you already have, but just in case):

"secureOptions": {
    "rejectUnauthorized": false
}
and FTPS should work.
Apart from that, make sure your hostname and port are set correctly.

If you're still unable to connect, there might be a server-side issue. In that case inspect the server logs to see what is happening.
brando90 commented 4 years ago

For completeness I saw this too but wasn't helpful: https://github.com/liximomo/vscode-sftp/issues/510

brando90 commented 4 years ago

One of the chats also had this link. I read it but I was confused what I was suppose to do with it:

https://github.com/liximomo/vscode-sftp/wiki/config#secure

brando90 commented 4 years ago

I confirmed with the administrators that the server logs do not show anything when I save my file.

Usually when I do rsync they see something.