search

Natizyskunk / vscode-sftp

Super fast sftp/ftp extension for VS Code
Other
397 stars 77 forks source link

Remote simlinks to directories are not followed #177

Open martenkoetsier opened 2 years ago

martenkoetsier commented 2 years ago

Describe the bug File upload does not follow remote simlinks. Instead, upload fails with message "[the dir] is not a valid directory path". Partial output of ls -l:

lrwxrwxrwx  1 user group     13 16 jun  2020 private_html -> ./public_html

(user and group redacted)

To Reproduce

  1. create some project with in it: 
    /test/file1.ext
/file2.ext
  2. setup sftp to connect to remote system
  3. on remote system, setup dir-structure: 
    /test2
simlink test -> test2
  4. in vs code, try to upload /file2.ext (works)
  5. in vs code, try to upload /test/file1.ext (does NOT work)

Expected behavior Simlinks to directories should be treated as "regular" directories, so that files can be uploaded to simlinked dirs.

Screenshots N/A

Desktop (please complete the following information):

Extension Logs from Startup - required

(various names and paths redacted)

[08-11 10:28:13] [info] config at /home/mk/Documents/dev/stichting-up/www/ 
 "remotePath":"./","uploadOnSave":false,"useTempFile":false,"openSsh":false,"downloadOnOpen":false,"ignore":[],"concurrency":4,"protocol":"ftp","connectTimeout":10000,"interactiveAuth":false,"secure":false,"remoteTimeOffsetInHours":0,"name":"connectionname","host":"hostname","port":21,"username":"******","password":"******","context":"www/","syncMode":"full","watcher":{"files":"*","autoUpload":false,"autoDelete":false}}
[08-11 10:41:48] [error] private_html (type = 3)is not a directory
[08-11 10:41:48] [error] Error: private_html is not a valid directory path
    at f.<anonymous> (/home/mk/.vscode/extensions/natizyskunk.sftp-1.15.14/dist/extension.js:2:225410)
    at Generator.next (<anonymous>)
    at s (/home/mk/.vscode/extensions/natizyskunk.sftp-1.15.14/dist/extension.js:2:222833)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) 
[08-11 10:58:38] [error] private_html (type = 3)is not a directory
[08-11 10:58:38] [error] Error: private_html is not a valid directory path
    at f.<anonymous> (/home/mk/.vscode/extensions/natizyskunk.sftp-1.15.14/dist/extension.js:2:225410)
    at Generator.next (<anonymous>)
    at s (/home/mk/.vscode/extensions/natizyskunk.sftp-1.15.14/dist/extension.js:2:222833)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
nerdx86 commented 2 years ago

I am having a similar issue with it not letting me upload files directly under a symbolic directory link (but files within subdirectories of the symlink work) It seems that files below the first "normal" directory inside of a directory symlink work but files directly under the symlink don't. (also have the same explorer issue as above where a directory symlink doesn't show as a directory)

I swear this worked until yesterday!!! My configuration files and some project documentation are directly under that symlink. LOL. I tried starting a different sshd with -d to a different port and I don't get anything interesting on the ubuntu side. Let me know what I can do to help figure this out?

I also use goodsync and winscp over sftp for the same directory structure and they works without issues.

Failed: [08-14 12:01:39] [info] [file-save] c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\blah.ini [08-14 12:01:39] [trace] handle upload file for c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\blah.ini [08-14 12:01:39] [debug] Outbound: Sending CHANNEL_DATA (r:0, 58) [08-14 12:01:39] [debug] SFTP: Outbound: Buffered MKDIR [08-14 12:01:39] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:01:39] [debug] SFTP: Inbound: Received STATUS (id:18, 4, "Failure") [08-14 12:01:39] [debug] Outbound: Sending CHANNEL_DATA (r:0, 54) [08-14 12:01:39] [debug] SFTP: Outbound: Buffered LSTAT [08-14 12:01:40] [debug] Inbound: CHANNEL_DATA (r:0, 41) [08-14 12:01:40] [debug] SFTP: Inbound: Received ATTRS (id:19) [08-14 12:01:40] [error] Error: Failure at 101 (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\SFTP.js:2618:19) at SFTP.push (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\SFTP.js:278:11) at CHANNEL_DATA (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\client.js:525:23) at 94 (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\handlers.misc.js:859:16) at Protocol.onPayload (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\Protocol.js:2025:10) at AESGCMDecipherNative.decrypt (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\crypto.js:987:26) at Protocol.parsePacket [as _parse] (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\Protocol.js:1994:25) at Protocol.parse (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\Protocol.js:293:16) at Socket. (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\client.js:713:21) at Socket.emit (node:events:390:28) at addChunk (node:internal/streams/readable:315:12) at readableAddChunk (node:internal/streams/readable:289:9) at Socket.Readable.push (node:internal/streams/readable:228:10) at TCP.onStreamRead (node:internal/stream_base_commons:199:23) download c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\blah.ini

and (cp on linux side of first working file below)

[08-14 12:22:21] [info] [file-save] c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\blah.cpp [08-14 12:22:21] [trace] handle upload file for c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\blah.cpp [08-14 12:22:21] [debug] Outbound: Sending CHANNEL_DATA (r:0, 58) [08-14 12:22:21] [debug] SFTP: Outbound: Buffered MKDIR [08-14 12:22:21] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:22:21] [debug] SFTP: Inbound: Received STATUS (id:54, 4, "Failure") [08-14 12:22:21] [debug] Outbound: Sending CHANNEL_DATA (r:0, 54) [08-14 12:22:21] [debug] SFTP: Outbound: Buffered LSTAT [08-14 12:22:21] [debug] Inbound: CHANNEL_DATA (r:0, 41) [08-14 12:22:21] [debug] SFTP: Inbound: Received ATTRS (id:55) [08-14 12:22:21] [error] Error: Failure at 101 (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\SFTP.js:2618:19) at SFTP.push (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\SFTP.js:278:11) at CHANNEL_DATA (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\client.js:525:23) at 94 (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\handlers.misc.js:859:16) at Protocol.onPayload (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\Protocol.js:2025:10) at AESGCMDecipherNative.decrypt (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\crypto.js:987:26) at Protocol.parsePacket [as _parse] (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\Protocol.js:1994:25) at Protocol.parse (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\protocol\Protocol.js:293:16) at Socket. (c:\Users\Sean.vscode\extensions\natizyskunk.sftp-1.15.14\node_modules\ssh2\lib\client.js:713:21) at Socket.emit (node:events:390:28) at addChunk (node:internal/streams/readable:315:12) at readableAddChunk (node:internal/streams/readable:289:9) at Socket.Readable.push (node:internal/streams/readable:228:10) at TCP.onStreamRead (node:internal/stream_base_commons:199:23) download c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\blah.cpp

Works: [08-14 12:12:17] [info] [file-save] c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\extensions\blah\blah.cpp [08-14 12:12:17] [trace] handle upload file for c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\extensions\blah\blah.cpp [08-14 12:12:17] [debug] Outbound: Sending CHANNEL_DATA (r:0, 86) [08-14 12:12:17] [debug] SFTP: Outbound: Buffered MKDIR [08-14 12:12:17] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:12:17] [debug] SFTP: Inbound: Received STATUS (id:24, 4, "Failure") [08-14 12:12:17] [debug] Outbound: Sending CHANNEL_DATA (r:0, 82) [08-14 12:12:17] [debug] SFTP: Outbound: Buffered LSTAT [08-14 12:12:17] [debug] Inbound: CHANNEL_DATA (r:0, 41) [08-14 12:12:17] [debug] SFTP: Inbound: Received ATTRS (id:25) [08-14 12:12:17] [debug] Outbound: Sending CHANNEL_DATA (r:0, 107) [08-14 12:12:17] [debug] SFTP: Outbound: Buffered OPEN [08-14 12:12:17] [debug] Inbound: CHANNEL_DATA (r:0, 17) [08-14 12:12:17] [debug] SFTP: Inbound: Received HANDLE (id:26) [08-14 12:12:17] [debug] Outbound: Sending CHANNEL_DATA (r:0, 17) [08-14 12:12:17] [debug] SFTP: Outbound: Buffered FSTAT [08-14 12:12:17] [debug] Inbound: CHANNEL_DATA (r:0, 41) [08-14 12:12:17] [debug] SFTP: Inbound: Received ATTRS (id:27) [08-14 12:12:17] [debug] Outbound: Sending CHANNEL_DATA (r:0, 25) [08-14 12:12:17] [debug] SFTP: Outbound: Buffered FSETSTAT [08-14 12:12:17] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:12:17] [debug] SFTP: Inbound: Received STATUS (id:28, 0, "Success") [08-14 12:12:17] [debug] Outbound: Sending CHANNEL_DATA (r:0, 16116) [08-14 12:12:17] [debug] SFTP: Outbound: Sent WRITE (id:29) [08-14 12:12:18] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:12:18] [debug] SFTP: Inbound: Received STATUS (id:29, 0, "Success") [08-14 12:12:18] [debug] Outbound: Sending CHANNEL_DATA (r:0, 29) [08-14 12:12:18] [debug] SFTP: Outbound: Buffered FSETSTAT [08-14 12:12:18] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:12:18] [debug] SFTP: Inbound: Received STATUS (id:30, 0, "Success") [08-14 12:12:18] [debug] Outbound: Sending CHANNEL_DATA (r:0, 17) [08-14 12:12:18] [debug] SFTP: Outbound: Buffered CLOSE [08-14 12:12:18] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:12:18] [debug] SFTP: Inbound: Received STATUS (id:31, 0, "Success") [08-14 12:12:18] [info] local ➞ remote c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\extensions\blah\blah.cpp

and (cp on linux side of first failed file above)

[08-14 12:20:34] [info] [file-save] c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\extensions\blah\blah.ini [08-14 12:20:34] [trace] handle upload file for c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\extensions\blah\blah.ini [08-14 12:20:34] [debug] Outbound: Sending CHANNEL_DATA (r:0, 86) [08-14 12:20:34] [debug] SFTP: Outbound: Buffered MKDIR [08-14 12:20:34] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:20:34] [debug] SFTP: Inbound: Received STATUS (id:46, 4, "Failure") [08-14 12:20:34] [debug] Outbound: Sending CHANNEL_DATA (r:0, 82) [08-14 12:20:34] [debug] SFTP: Outbound: Buffered LSTAT [08-14 12:20:34] [debug] Inbound: CHANNEL_DATA (r:0, 41) [08-14 12:20:34] [debug] SFTP: Inbound: Received ATTRS (id:47) [08-14 12:20:34] [debug] Outbound: Sending CHANNEL_DATA (r:0, 106) [08-14 12:20:34] [debug] SFTP: Outbound: Buffered OPEN [08-14 12:20:34] [debug] Inbound: CHANNEL_DATA (r:0, 17) [08-14 12:20:34] [debug] SFTP: Inbound: Received HANDLE (id:48) [08-14 12:20:34] [debug] Outbound: Sending CHANNEL_DATA (r:0, 17) [08-14 12:20:34] [debug] SFTP: Outbound: Buffered FSTAT [08-14 12:20:34] [debug] Inbound: CHANNEL_DATA (r:0, 41) [08-14 12:20:34] [debug] SFTP: Inbound: Received ATTRS (id:49) [08-14 12:20:34] [debug] Outbound: Sending CHANNEL_DATA (r:0, 25) [08-14 12:20:34] [debug] SFTP: Outbound: Buffered FSETSTAT [08-14 12:20:34] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:20:34] [debug] SFTP: Inbound: Received STATUS (id:50, 0, "Success") [08-14 12:20:34] [debug] Outbound: Sending CHANNEL_DATA (r:0, 5391) [08-14 12:20:34] [debug] SFTP: Outbound: Sent WRITE (id:51) [08-14 12:20:34] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:20:34] [debug] SFTP: Inbound: Received STATUS (id:51, 0, "Success") [08-14 12:20:34] [debug] Outbound: Sending CHANNEL_DATA (r:0, 29) [08-14 12:20:34] [debug] SFTP: Outbound: Buffered FSETSTAT [08-14 12:20:35] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:20:35] [debug] SFTP: Inbound: Received STATUS (id:52, 0, "Success") [08-14 12:20:35] [debug] Outbound: Sending CHANNEL_DATA (r:0, 17) [08-14 12:20:35] [debug] SFTP: Outbound: Buffered CLOSE [08-14 12:20:35] [debug] Inbound: CHANNEL_DATA (r:0, 28) [08-14 12:20:35] [debug] SFTP: Inbound: Received STATUS (id:53, 0, "Success") [08-14 12:20:35] [info] local ➞ remote c:\repo_blah\blah\rootpath_blah\symlink_to_dir_blah\extensions\blah\blah.ini

Paulo--M-- commented 2 years ago

I confirm that symlinks are not being followed. Is there any fix for this behaviour?

Natizyskunk commented 2 years ago

guys can you try this please ?

https://github.com/Natizyskunk/vscode-sftp/blob/master/FAQ.md#error-failure---solution-one https://github.com/Natizyskunk/vscode-sftp/blob/master/FAQ.md#how-do-i-upload-content-inside-a-folder-but-not-the-folder-itself using Context like this https://github.com/Natizyskunk/vscode-sftp#multiple-context (also some help here https://github.com/liximomo/vscode-sftp/issues/107#issuecomment-698089487

Sorry that I'm actually not able to help you more with this bug :/

nerdx86 commented 2 years ago

Ok, so you are suggesting that instead of using symbolic links for the directories, that we create separate "contexts" on the VSCode side that creates "client" side symlinks? Ok that might work, I'll give it a shot when I'm back in code...

it's actually kinda odd that we can read but not write.. I wonder if it's a server side issue at the root cause? WinSCP can handle it, but they are definitely doing extra operations (they actually can display the links, etc) GoodSync does as well.

Here's the detailed use case that I have as an FYI: My situation is that I am working with NVIDIA Deepstream development on a Linux platform from my Windows Development machine. Their build environment uses Bazel, so it's kinda a mess of dependencies from http, github, etc as well as a lot of raw linux dependencies. To get VSCode to be able to handle that at all I created a folder of symbolic links to all the includes, sources, and libraries necessary (in the "shape" necessary because of the bazel process which is different than where the files actually reside on disk (thanks Bazel, but that's also part of NVIDIA's package and not a good idea to work around it for future compatibility).

Full disclosure: I really am not a VSCode fan, I prefer the idea of a project file that separates the file system from the project structure BUT VSCode has your plugin which makes it VERY helpful for editing on windows and building on linux. (actually editing on windows 2 time zones away from the linux build) Thanks for the great tool.. I tried to look into the code to see if I could help, but I'm an old geezer and it looked like it was in typescript which is something that I hadn't used before and figured chances were good that even if I figured something out I'd probably not do a very good job.. :)

Thank You!!!!!!

On Mon, Sep 12, 2022 at 4:00 PM Natan FOURIÉ @.***> wrote:

guys can you try this please ?

https://github.com/Natizyskunk/vscode-sftp/blob/master/FAQ.md#error-failure---solution-one

https://github.com/Natizyskunk/vscode-sftp/blob/master/FAQ.md#how-do-i-upload-content-inside-a-folder-but-not-the-folder-itself using Context https://github.com/Natizyskunk/vscode-sftp/wiki/config#context like this https://github.com/Natizyskunk/vscode-sftp#multiple-context (also some help here liximomo#107 (comment) https://github.com/liximomo/vscode-sftp/issues/107#issuecomment-698089487

Sorry that I'm actually not able to help you more with this bug :/

— Reply to this email directly, view it on GitHub https://github.com/Natizyskunk/vscode-sftp/issues/177#issuecomment-1244326306, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQGRWBXTMHRZKNQYAQRNL3V56DXVANCNFSM56HOMC2Q . You are receiving this because you commented.Message ID: @.***>

Natizyskunk commented 2 years ago

Related issue : #60

nocodehummel commented 1 year ago

The issue also occurs on my system with VSC 1.73.1 with SFTP 1.16.1.

Upload fails when the uploaded file is directly under the symlinked folder. Files in a sub-folder are uploaded without problems.

Context reference(s) to multiple source folders on the remote is a good workaround.

Kesuaheli commented 9 months ago

+1 We added some sort of system to our files on the remote, to have easier access with sftp clients. This involves symlinks which we use to accomplish a universial autolink config. But I noticed, that this extention does not support symlinks. And found this issue already exists. While a host connection with

{
  ...
  "remotePath": "/path/to/myLink/",
  ...
}

correctly connects to the directory pointed to by myLink, a connection to lets say

{
  ...
  "remotePath": "/path/to/",
  ...
}

(while that folder still holds our myLink) does not follow the link at all (on read). On uploads/write operations it only works if you upload a file to myLink/anotherFolder/file.txt but not to myLink/directFile.txt without a subfolder after the link.

The point of adding multiple hosts for each symlink kinda defeats our system to have symlinks at all: having a universial config that is able to autolink itself depending on your local setup. I would appreciate it, if symlinks are fully supported.