search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
163.3k stars 28.9k forks source link

VSCode DDoS itself discovering ports for the tunnel feature #230738

Open jeremie-stripe opened 2 days ago

jeremie-stripe commented 2 days ago

Does this issue occur when all extensions are disabled?: Not applicable

Steps to Reproduce:

  1. Have a running process with a long (.... very long) command line and a dozen open TCP ports
  2. Open VSCode with remote dev

Yeah so this is a fun one.

We have machines where we run VSCode that also happens to be running Java processes with ridiculously long command lines (classpaths and all). When I say long I literally mean command lines with > 100K characters. Those processes also end up opening several TCP listening ports (> 10).

What we have noticed is that when starting a new VSCode session, the extension host process would literally sit there completely stalled out for a few minutes. Extension loading request would go unanswered and the extension host process would go at 100% CPU in htop. Then at some point it would unstick itself and proceed as normal.

After playing with prof and Node.js built-in --prof, the culprit turned out to be surprising:

Statistical profiling result from isolate-0x750d9b0-1665554-v8.log, (214558 ticks, 65 unaccounted, 0 excluded).

 [Shared libraries]:
   ticks  total  nonlib   name
   1275    0.6%          /tmp/vscode-server-2326317938/node
     39    0.0%          /usr/lib/x86_64-linux-gnu/libc.so.6
      4    0.0%          /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.33

 [JavaScript]:
   ticks  total  nonlib   name
  212869   99.2%   99.8%  RegExp: .*\.vscode-server-[a-zA-Z]+\/bin.*
      5    0.0%    0.0%  RegExp: \s+
      1    0.0%    0.0%  JS: ^write /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:115:12598
      1    0.0%    0.0%  JS: ^normalize node:path:1130:12
      1    0.0%    0.0%  JS: ^isEmpty node:internal/fixed_queue:67:10
      1    0.0%    0.0%  JS: ^dirname node:path:1278:10
      1    0.0%    0.0%  JS: ^c /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:115:9649
      1    0.0%    0.0%  JS: ^S /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:158:5780
      1    0.0%    0.0%  JS: ^O /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:105:4193
      1    0.0%    0.0%  JS: ^Module._resolveFilename node:internal/modules/cjs/loader:1059:35
      1    0.0%    0.0%  JS: ^Module._extensions..js node:internal/modules/cjs/loader:1370:37
      1    0.0%    0.0%  JS: *wrappedFn node:internal/errors:536:21
      1    0.0%    0.0%  JS: *normalizeString node:path:66:25
      1    0.0%    0.0%  JS: *acceptChunk /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:115:11536

 [C++]:
   ticks  total  nonlib   name
    111    0.1%    0.1%  epoll_pwait@@GLIBC_2.6
     60    0.0%    0.0%  fwrite@@GLIBC_2.2.5
     51    0.0%    0.0%  __write@@GLIBC_2.2.5
     12    0.0%    0.0%  std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long)@@GLIBCXX_3.4.9
     12    0.0%    0.0%  _IO_file_xsputn@@GLIBC_2.2.5
     10    0.0%    0.0%  std::ostream::sentry::sentry(std::ostream&)@@GLIBCXX_3.4
      5    0.0%    0.0%  __read@@GLIBC_2.2.5
      5    0.0%    0.0%  __libc_malloc@@GLIBC_2.2.5
      4    0.0%    0.0%  isprint@@GLIBC_2.2.5
      3    0.0%    0.0%  __pthread_mutex_lock@GLIBC_2.2.5
      2    0.0%    0.0%  std::ostreambuf_iterator<char, std::char_traits<char> > std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<long>(std::ostreambuf_iterator<char, std::char_traits<char> >, std::ios_base&, char, long) const@@GLIBCXX_3.4
      2    0.0%    0.0%  std::ostream& std::ostream::_M_insert<long>(long)@@GLIBCXX_3.4.9
      2    0.0%    0.0%  std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@@GLIBCXX_3.4
      2    0.0%    0.0%  __munmap@@GLIBC_PRIVATE
      1    0.0%    0.0%  std::ostream::operator<<(int)@@GLIBCXX_3.4
      1    0.0%    0.0%  std::_Rb_tree_decrement(std::_Rb_tree_node_base*)@@GLIBCXX_3.4
      1    0.0%    0.0%  pthread_cond_signal@@GLIBC_2.3.2
      1    0.0%    0.0%  operator new(unsigned long)@@GLIBCXX_3.4
      1    0.0%    0.0%  __pthread_mutex_unlock@GLIBC_2.2.5
      1    0.0%    0.0%  __mprotect@@GLIBC_PRIVATE
      1    0.0%    0.0%  __madvise@@GLIBC_PRIVATE
      1    0.0%    0.0%  __lll_lock_wait_private@@GLIBC_PRIVATE

 [Summary]:
   ticks  total  nonlib   name
  212886   99.2%   99.8%  JavaScript
    289    0.1%    0.1%  C++
     66    0.0%    0.0%  GC
   1318    0.6%          Shared libraries
     65    0.0%          Unaccounted

 [C++ entry points]:
   ticks    cpp   total   name
     58   36.9%    0.0%  fwrite@@GLIBC_2.2.5
     44   28.0%    0.0%  __write@@GLIBC_2.2.5
     12    7.6%    0.0%  std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long)@@GLIBCXX_3.4.9
     12    7.6%    0.0%  _IO_file_xsputn@@GLIBC_2.2.5
     10    6.4%    0.0%  std::ostream::sentry::sentry(std::ostream&)@@GLIBCXX_3.4
      5    3.2%    0.0%  __libc_malloc@@GLIBC_2.2.5
      4    2.5%    0.0%  isprint@@GLIBC_2.2.5
      3    1.9%    0.0%  __pthread_mutex_lock@GLIBC_2.2.5
      2    1.3%    0.0%  std::ostreambuf_iterator<char, std::char_traits<char> > std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<long>(std::ostreambuf_iterator<char, std::char_traits<char> >, std::ios_base&, char, long) const@@GLIBCXX_3.4
      2    1.3%    0.0%  std::ostream& std::ostream::_M_insert<long>(long)@@GLIBCXX_3.4.9
      2    1.3%    0.0%  std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@@GLIBCXX_3.4
      1    0.6%    0.0%  std::ostream::operator<<(int)@@GLIBCXX_3.4
      1    0.6%    0.0%  std::_Rb_tree_decrement(std::_Rb_tree_node_base*)@@GLIBCXX_3.4
      1    0.6%    0.0%  operator new(unsigned long)@@GLIBCXX_3.4

 [Bottom up (heavy) profile]:
  Note: percentage shows a share of a particular caller in the total
  amount of its parent calls.
  Callers occupying less than 1.0% are not shown.

   ticks parent  name
  212869   99.2%  RegExp: .*\.vscode-server-[a-zA-Z]+\/bin.*
  212869  100.0%    /tmp/vscode-server-2326317938/node
  193767   91.0%      JS: ~y /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:183:164
  193767  100.0%        JS: ~<anonymous> /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:184:275
  193767  100.0%          /tmp/vscode-server-2326317938/node
  193767  100.0%            JS: ~k /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:184:202
  19102    9.0%      /tmp/vscode-server-2326317938/node
  19102  100.0%        JS: ~y /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:183:164
  19102  100.0%          JS: ~<anonymous> /tmp/vscode-server-2326317938/out/vs/workbench/api/node/extensionHostProcess.js:184:275
  19102  100.0%            /tmp/vscode-server-2326317938/node

The important part is: 212869 99.2% RegExp: .*\.vscode-server-[a-zA-Z]+\/bin.*

This regex is from https://github.com/microsoft/vscode/blob/71f16d9c134289a9d2f1951faab4183eab41b59f/src/vs/workbench/api/node/extHostTunnelService.ts#L107-L111 (the ~y minified symbol in the profile trace maps to knownExcludeCmdline)

The explanation seems to be that because VSCode will initialize this service eagerly when the EH process is started and run the discovery process eagerly, it will:

End result: it stalls out

I did not see a way to disable this functionality by CLI arguments nor by settings, since we don't need it and we may be a special case, having one would solve the problem.

vs-code-engineering[bot] commented 2 days ago

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.94.0. Please try upgrading to the latest version and checking whether this issue remains.

Happy Coding!

alexr00 commented 1 day ago

Thanks for the investigation! I'll add a check in there so we don't try to look at command lines that are so long.

You can turn off port discovery by setting "remote.autoForwardPortsSource": "output". We won't start watching the proc filesystem at all if that is set.