Remote server disconnected when delete some files

Bo520 commented 3 years ago

OS:Windows 10 20H2 I use some extensions to connect my remote CentOS server. However, connection always be disconnected when I delete files, just like this figure. Reenter password didn't work, I find that I must restart vscode. Files will be deleted when I restart vscode and reconnect my remote server.

roblourens commented 3 years ago

Is there something like a build process that might start using lots of CPU when files are deleted?
Does it happen when you delete fewer files at a time?
Do you have any remote extensions, and does disabling them help?

And on the issue of the password prompt: Can you share the full Remote-SSH log? Does it ask for your password when you initially connect and that works?

Bo520 commented 3 years ago

@roblourens Thanks for reply, I hope the following information may be helpful for you.

Important info: My remote-server runs on campus network, so I use VPN provided by school to connect it. I use this VPN app: However, business as usual when I do the same operations on another remote server which has a public IP. I can't find any bug if I connect a remote server directly. I can delete few files and lots of files. So, I think the key to this problem is the VPN.
- Is there something like a build process that might start using lots of CPU when files are deleted?
No, the CPU utilization rate only increased from 6% to about 12%.
- Does it happen when you delete fewer files at a time?
I recently find that is the file sizes that cause this problem. Business as usual when I delete some small files but connection will be disconnected when I delete a big file.
- Do you have any remote extensions, and does disabling them help?
I have this remote extensions: Remote - Containers, Remote - SSH, Remote - SSH: Editing Configuration Files, Remote - WSL, Remote Development, Remote X11 (SSH). I think only the last one may have something to do with this problem so I uninstall it. However, this problem still exists.

And on the issue of the password prompt: Can you share the full Remote-SSH log? Does it ask for your password when you initially connect and that works?
Sorry, I don't know where and how to export the Remote-SSH log. When I start vscode and initially connect it will ask me for password and I can login in successfuly. If the problem happened and connection disconnected it will always ask me for password but it doesn't work. I canonly restart vscode to reconnect my remote server.

roblourens commented 3 years ago

We saw something like this in https://github.com/microsoft/vscode-remote-release/issues/3184, it went away, my guess is that the connection is slower over the VPN, but I still don't know what traffic is produced just by deleting a file. cc @alexdima

Sorry, I don't know where and how to export the Remote-SSH log

In the same panel where you see the Terminal, click Output, then pick Remote-SSH from the dropdown.

alexdima commented 3 years ago

cc @isidorn I think we read files that are being deleted from the explorer for explorer undo. We might want to limit the file size when being connected to an ssh.

isidorn commented 3 years ago

@alexdima we can introduce a limit, but what is a good limit? I don't want to make this configurable at first. Maybe 1mb?

Bo520 commented 3 years ago

@isidorn @isidorn @roblourens I take an video to show the problem I meet and I upload it to my github repository because I can't upload an video bigger than 20 m in this place.

https://github.com/Bo520/BoImgStore/blob/main/vscode_bug.mp4

A full log in that video, from I start vscode, login my remote server, delete a big file to I reenter password:

[21:10:47.282] Log Level: 2
[21:10:47.284] remote-ssh@0.64.0
[21:10:47.285] win32 x64
[21:10:47.285] SSH Resolver called for "ssh-remote+10.130.35.2", attempt 1
[21:10:47.285] "remote.SSH.useLocalServer": false
[21:10:47.286] "remote.SSH.showLoginTerminal": false
[21:10:47.286] "remote.SSH.remotePlatform": {"GinPie":"linux","10.130.35.2":"linux","remote.ginpie.com":"linux","127.0.0.1":"linux"}
[21:10:47.286] "remote.SSH.sshPath": undefined
[21:10:47.286] "remote.SSH.sshConfigurationFile": undefined
[21:10:47.286] "remote.SSH.useFlock": true
[21:10:47.286] "remote.SSH.lockfilesInTmp": false
[21:10:47.287] "remote.SSH.localServerDownload": auto
[21:10:47.287] "remote.SSH.remoteServerListenOnSocket": false
[21:10:47.287] "remote.SSH.showLoginTerminal": false
[21:10:47.287] "remote.SSH.defaultExtensions": []
[21:10:47.287] SSH Resolver called for host: 10.130.35.2
[21:10:47.287] Setting up SSH remote "10.130.35.2"
[21:10:47.299] Using commit id "622cb03f7e070a9670c94bae1a45d78d7181fbd4" and quality "stable" for server
[21:10:47.302] Install and start server if needed
[21:10:47.306] Checking ssh with "ssh -V"
[21:10:47.337] > OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5

[21:10:47.341] Running script with connection command: ssh -T -D 52861 "10.130.35.2" bash
[21:10:47.346] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[21:10:47.582] > fosu@10.130.35.2's password:]0;C:\WINDOWS\System32\cmd.exe
[21:10:47.583] Got some output, clearing connection timeout
[21:10:47.584] Showing password prompt
[21:10:53.080] Got password response
[21:10:53.081] "install" wrote data to terminal: "********"
[21:10:53.095] > 
[21:10:54.094] > 0a5077c8931b: running
[21:10:54.192] > Acquiring lock on /home/fosu/.vscode-server/bin/622cb03f7e070a9670c94bae1a45d78d
> 7181fbd4/vscode-remote-lock.fosu.622cb03f7e070a9670c94bae1a45d78d7181fbd4       
[21:10:54.224] > Found existing installation at /home/fosu/.vscode-server/bin/622cb03f7e070a9670c
> 94bae1a45d78d7181fbd4...
> Checking /home/fosu/.vscode-server/.622cb03f7e070a9670c94bae1a45d78d7181fbd4.log
>  and /home/fosu/.vscode-server/.622cb03f7e070a9670c94bae1a45d78d7181fbd4.pid for
>  a running server
> Looking for server with pid: 32558
[21:10:54.334] > Starting server with command... /home/fosu/.vscode-server/bin/622cb03f7e070a9670
> c94bae1a45d78d7181fbd4/server.sh --start-server --host=127.0.0.1 --enable-remote
> -auto-shutdown  --port=0  &> "/home/fosu/.vscode-server/.622cb03f7e070a9670c94ba
> e1a45d78d7181fbd4.log" < /dev/null
> printenv:
[21:10:54.340] >     MKLROOT=/opt/intel/compilers_and_libraries_2018.5.274/linux/mkl
>     MANPATH=/opt/intel/man/common:/opt/intel/compilers_and_libraries_2018.5.274/
> linux/mpi/man:/opt/intel/documentation_2018/en/debugger//gdb-ia/man/:/opt/intel/
> documentation_2018/en/debugger//gdb-igfx/man/:/usr/local/share/man:/usr/share/ma
> n/overrides:/usr/share/man:
>     XDG_SESSION_ID=8042
>     SELINUX_ROLE_REQUESTED=
>     IPPROOT=/opt/intel/compilers_and_libraries_2018.5.274/linux/ipp
>     INTEL_LICENSE_FILE=/opt/intel/compilers_and_libraries_2018.5.274/linux/licen
> ses:/opt/intel/licenses:/home/fosu/intel/licenses
>     SHELL=/bin/bash
>     SSH_CLIENT=10.178.0.27 52866 22
>     GDBSERVER_MIC=/opt/intel/debugger_2018/gdb/targets/intel64/x200/bin/gdbserve
> r
>     LIBRARY_PATH=/opt/intel/compilers_and_libraries_2018.5.274/linux/ipp/lib/int
> el64:/opt/intel/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_li
> n:/opt/intel/compilers_and_libraries_2018.5.274/linux/mkl/lib/intel64_lin:/opt/i
> ntel/compilers_and_libraries_2018.5.274/linux/tbb/lib/intel64/gcc4.7:/opt/intel/
> compilers_and_libraries_2018.5.274/linux/tbb/lib/intel64/gcc4.7:/opt/intel/compi
> lers_and_libraries_2018.5.274/linux/daal/lib/intel64_lin:/opt/intel/compilers_an
> d_libraries_2018.5.274/linux/daal/../tbb/lib/intel64_lin/gcc4.4
>     CONDA_SHLVL=1
[21:10:54.347] > 
>     SELINUX_USE_CURRENT_RANGE=
>     CONDA_PROMPT_MODIFIER=(base)
>     USER=fosu
>     LD_LIBRARY_PATH=/usr/local/cuda/lib64:/opt/intel/mkl/lib/intel64:/opt/intel/
> impi/2018.4.274/lib64:/opt/intel/lib:/opt/intel/compilers_and_libraries_2018.5.2
> 74/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2018.5.274/linu
> x/compiler/lib/intel64_lin:/opt/intel/compilers_and_libraries_2018.5.274/linux/m
> pi/intel64/lib:/opt/intel/compilers_and_libraries_2018.5.274/linux/mpi/mic/lib:/
> opt/intel/compilers_and_libraries_2018.5.274/linux/ipp/lib/intel64:/opt/intel/co
> mpilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin:/opt/intel/compi
> lers_and_libraries_2018.5.274/linux/mkl/lib/intel64_lin:/opt/intel/compilers_and
> _libraries_2018.5.274/linux/tbb/lib/intel64/gcc4.7:/opt/intel/compilers_and_libr
> aries_2018.5.274/linux/tbb/lib/intel64/gcc4.7:/opt/intel/debugger_2018/iga/lib:/
> opt/intel/debugger_2018/libipt/intel64/lib:/opt/intel/compilers_and_libraries_20
> 18.5.274/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2018.5.27
> 4/linux/daal/../tbb/lib/intel64_lin/gcc4.4
>     PSTLROOT=/opt/intel/compilers_and_libraries_2018.5.274/linux/pstl
>     CONDA_EXE=/opt/anaconda3/bin/conda
>     CPATH=/opt/intel/compilers_and_libraries_2018.5.274/linux/ipp/include:/opt/i
> ntel/compilers_and_libraries_2018.5.274/linux/mkl/include:/opt/intel/compilers_a
> nd_libraries_2018.5.274/linux/pstl/include:/opt/intel/compilers_and_libraries_20
> 18.5.274/linux/tbb/include:/opt/intel/compilers_and_libraries_2018.5.274/linux/t
> bb/include:/opt/intel/compilers_and_libraries_2018.5.274/linux/daal/include     
>     VSCODE_AGENT_FOLDER=/home/fosu/.vscode-server
>     _CE_CONDA=
>     NLSPATH=/opt/intel/compilers_and_libraries_2018.5.274/linux/compiler/lib/int
> el64/locale/%l_%t/%N:/opt/intel/compilers_and_libraries_2018.5.274/linux/mkl/lib
> /intel64_lin/locale/%l_%t/%N:/opt/intel/debugger_2018/gdb/intel64/share/locale/%
> l_%t/%N
[21:10:54.352] > 
>     PATH=/home/fosu/si1_rescu:/opt/anaconda3/bin:/opt/anaconda3/condabin:/home/f
> osu/g16:/home/fosu/gv:/home/fosu/vaspkit.1.2.4/bin:/usr/local/cuda/bin:/opt/inte
> l/impi/2018.4.274/bin64:/opt/intel/bin:/opt/intel/compilers_and_libraries_2018.5
> .274/linux/bin/intel64:/opt/intel/compilers_and_libraries_2018.5.274/linux/mpi/i
> ntel64/bin:/usr/local/bin:/usr/bin
>     MAIL=/var/mail/fosu
>     TBBROOT=/opt/intel/compilers_and_libraries_2018.5.274/linux/tbb
>     CONDA_PREFIX=/opt/anaconda3
>     PWD=/home/fosu
>     LANG=zh_CN.UTF-8
>     GDB_CROSS=/opt/intel/debugger_2018/gdb/intel64/bin/gdb-ia
>     GAUSS_EXEDIR=/home/fosu/g16
>     SELINUX_LEVEL_REQUESTED=
>     DAALROOT=/opt/intel/compilers_and_libraries_2018.5.274/linux/daal
>     MPM_LAUNCHER=/opt/intel/debugger_2018/mpm/mic/bin/start_mpm.sh
>     _CE_M=
>     INTEL_PYTHONHOME=/opt/intel/debugger_2018/python/intel64/
>     HOME=/home/fosu
>     SHLVL=2
>     LOGNAME=fosu
>     CONDA_PYTHON_EXE=/opt/anaconda3/bin/python
>     SSH_CONNECTION=10.178.0.27 52866 10.130.35.2 22
[21:10:54.358] > 
>     XDG_DATA_DIRS=/home/fosu/.local/share/flatpak/exports/share:/var/lib/flatpak
> /exports/share:/usr/local/share:/usr/share
>     CLASSPATH=/opt/intel/compilers_and_libraries_2018.5.274/linux/mpi/intel64/li
> b/mpi.jar:/opt/intel/compilers_and_libraries_2018.5.274/linux/daal/lib/daal.jar 
>     LESSOPEN=||/usr/bin/lesspipe.sh %s
>     PKG_CONFIG_PATH=/opt/intel/compilers_and_libraries_2018.5.274/linux/mkl/bin/
> pkgconfig
>     CONDA_DEFAULT_ENV=base
>     INFOPATH=/opt/intel/documentation_2018/en/debugger//gdb-ia/info/:/opt/intel/
> documentation_2018/en/debugger//gdb-igfx/info/
>     XDG_RUNTIME_DIR=/run/user/1001
>     I_MPI_ROOT=/opt/intel/compilers_and_libraries_2018.5.274/linux/mpi
>     _=/usr/bin/printenv
> Spawned remote server: 34056
[21:10:54.367] > 
> Waiting for server log...
[21:10:54.374] > 
[21:10:54.408] > Waiting for server log...
[21:10:54.450] > Waiting for server log...
[21:10:54.500] >  
> *
> * Reminder: You may only use this software with Visual Studio family products,  
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057) 
> *
> 
[21:10:54.511] > 
[21:10:54.523] > 0a5077c8931b: start
> sshAuthSock====
> webUiAccessToken====
> listeningOn==45264==
> osReleaseId==centos==
> arch==x86_64==
> tmpDir==/run/user/1001==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime==155==
> connectionToken==a11aa111-1111-1aa1-1aa1-11a11aa1a11a==
> 0a5077c8931b: end
[21:10:54.524] Received install output: 
sshAuthSock====
webUiAccessToken====
listeningOn==45264==
osReleaseId==centos==
arch==x86_64==
tmpDir==/run/user/1001==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime==155==
connectionToken==a11aa111-1111-1aa1-1aa1-11a11aa1a11a==

[21:10:54.525] Remote server is listening on 45264
[21:10:54.525] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":45264},"osReleaseId":"centos","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","tmpDir":"/run/user/1001","platform":"linux","connectionToken":"a11aa111-1111-1aa1-1aa1-11a11aa1a11a"},"serverStartTime":155,"installUnpackCode":""}
[21:10:54.526] Starting forwarding server. localPort 52874 -> socksPort 52861 -> remotePort 45264
[21:10:54.527] Forwarding server listening on 52874
[21:10:54.527] Waiting for ssh tunnel to be ready
[21:10:54.529] [Forwarding server 52874] Got connection 0
[21:10:54.530] Tunneled 45264 to local port 52874
[21:10:54.530] Resolved "ssh-remote+10.130.35.2" to "127.0.0.1:52874"
[21:10:54.539] ------

[21:10:54.574] > 
[21:10:54.574] [Forwarding server 52874] Got connection 1
[21:10:54.684] [Forwarding server 52874] Got connection 2
[21:10:56.015] [Forwarding server 52874] Got connection 3
[21:10:56.016] [Forwarding server 52874] Got connection 4
[21:10:56.023] [Forwarding server 52874] Got connection 5
[21:10:57.180] Unhandled rejection: Error: spawn C:\Program Files\Git\cmd\git.exe ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
    at onErrorNT (internal/child_process.js:469:16)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)
[21:10:57.187] Unhandled rejection: Error: spawn C:\Program Files\Git\cmd\git.exe ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:267:19)
    at onErrorNT (internal/child_process.js:469:16)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)
[21:11:26.673] ------

[21:11:26.673] SSH Resolver called for "ssh-remote+10.130.35.2", attempt 2, (Reconnection)
[21:11:26.673] SSH Resolver called for host: 10.130.35.2
[21:11:26.673] Setting up SSH remote "10.130.35.2"
[21:11:26.676] Using commit id "622cb03f7e070a9670c94bae1a45d78d7181fbd4" and quality "stable" for server
[21:11:26.681] Install and start server if needed
[21:11:26.687] Running script with connection command: ssh -T -D 53222 "10.130.35.2" bash
[21:11:26.692] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[21:11:26.930] > fosu@10.130.35.2's password:]0;C:\WINDOWS\System32\cmd.exe
[21:11:26.930] Got some output, clearing connection timeout
[21:11:26.931] Showing password prompt
[21:11:35.150] Got password response
[21:11:35.150] "install" wrote data to terminal: "********"
[21:11:35.162] > 
[21:11:36.123] > 156cce06a94f: running
[21:11:36.213] > Acquiring lock on /home/fosu/.vscode-server/bin/622cb03f7e070a9670c94bae1a45d78d
> 7181fbd4/vscode-remote-lock.fosu.622cb03f7e070a9670c94bae1a45d78d7181fbd4       
[21:11:36.242] > Found existing installation at /home/fosu/.vscode-server/bin/622cb03f7e070a9670c
> 94bae1a45d78d7181fbd4...
[21:11:36.257] > Checking /home/fosu/.vscode-server/.622cb03f7e070a9670c94bae1a45d78d7181fbd4.log
>  and /home/fosu/.vscode-server/.622cb03f7e070a9670c94bae1a45d78d7181fbd4.pid for
>  a running server
> Looking for server with pid: 34056
[21:11:36.368] > Found running server...
[21:11:36.377] > 
> *
> * Reminder: You may only use this software with Visual Studio family products, 
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
> 
[21:11:36.407] > Checking server status on port 45264 with wget
[21:11:36.434] > 156cce06a94f: start  
> sshAuthSock====      
> webUiAccessToken==== 
> listeningOn==45264== 
> osReleaseId==centos==
> arch==x86_64==       
> tmpDir==/run/user/1001==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==a11aa111-1111-1aa1-1aa1-11a11aa1a11a==
> 156cce06a94f: end
[21:11:36.434] Received install output:   
sshAuthSock====      
webUiAccessToken==== 
listeningOn==45264== 
osReleaseId==centos==
arch==x86_64==       
tmpDir==/run/user/1001==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==a11aa111-1111-1aa1-1aa1-11a11aa1a11a==

[21:11:36.435] Remote server is listening on 45264
[21:11:36.435] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":45264},"osReleaseId":"centos","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","tmpDir":"/run/user/1001","platform":"linux","connectionToken":"a11aa111-1111-1aa1-1aa1-11a11aa1a11a"},"installUnpackCode":""}
[21:11:36.436] Starting forwarding server. localPort 53229 -> socksPort 53222 -> remotePort 45264
[21:11:36.437] Forwarding server listening on 53229
[21:11:36.437] Waiting for ssh tunnel to be ready
[21:11:36.439] [Forwarding server 53229] Got connection 0
[21:11:36.440] Tunneled 45264 to local port 53229
[21:11:36.440] Resolved "ssh-remote+10.130.35.2" to "127.0.0.1:53229"
[21:11:36.445] ------

[21:11:36.452] [Forwarding server 53229] Got connection 1
[21:11:56.960] ------

[21:11:56.960] SSH Resolver called for "ssh-remote+10.130.35.2", attempt 3, (Reconnection)
[21:11:56.961] SSH Resolver called for host: 10.130.35.2
[21:11:56.961] Setting up SSH remote "10.130.35.2"
[21:11:56.963] Using commit id "622cb03f7e070a9670c94bae1a45d78d7181fbd4" and quality "stable" for server
[21:11:56.966] Install and start server if needed
[21:11:56.973] Running script with connection command: ssh -T -D 53249 "10.130.35.2" bash
[21:11:56.977] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[21:11:57.424] > fosu@10.130.35.2's password:]0;C:\WINDOWS\System32\cmd.exe
[21:11:57.424] Got some output, clearing connection timeout
[21:11:57.425] Showing password prompt

roblourens commented 3 years ago

I had no idea we do that, is that a new feature? Can we leave the file content in the remote EH instead of moving it around?

I don't think I would trust this undo feature in a remote, tbh, it's nice if it works but when I see the "permanently delete" dialog I have to assume the file is just gone.

yonghochang77 commented 8 months ago

I also had this problem and lost the source code twice. I'm saving it in git, but I'm scared that if the connection is lost, the data will be lost. I can't use git stash because the terminal hangs up. Working locally is difficult because testing is not possible because the target environment is different. Must work remotely and not be deleted. I thought someone on my team hated me...

microsoft / vscode-remote-release

Remote server disconnected when delete some files #4504