search

Shopify / ruby-lsp

An opinionated language server for Ruby
https://shopify.github.io/ruby-lsp/
MIT License
1.51k stars 140 forks source link

Running rails tests in the vscode test explorer is quite slow #2402

Open andypeters opened 1 month ago

andypeters commented 1 month ago

Description:

When running a test class in the VScode Test Explorer, the tests take quite a while to complete. It seems they are running one at a time starting with the last test first (visually from the bottom up). Additionally, it indicates running in a single process per test.

If this is known or expected, I'm sorry I missed the mention. I scoped out the issue list here and shopify/ruby-lsp-rails as best I could. There were a few somewhat related regarding speed to rspec tests, but I know that is an extension so I thought it fitting to report in regard to minitest here. One being this: https://github.com/Shopify/ruby-lsp/issues/2049

Reproduction steps

  1. Open any rails project in VScode.
  2. Select a test. ie: test/models/foo_test.
  3. Wait for Run / Run in Terminal / Debug appear.
  4. Open Test Explorer from the sidebar. Wait for the class to be visible (ie step three of course).
  5. Either click the "Play icon" next to the test class or click "Run" in the editor. (I have gifs illustrating below)
  6. Observe the tests run one at a time. Additionally, watch "Run in Terminal" zip by.

test_helper.rb:

parallelize(workers: :number_of_processors)

Grasping at straws, I went bonkers and nuclear 💥

Gifs for ya from a throw away rails app

Test Explorer Console
test-explorer console-test-output

Screenshot for "running in a single process"

Screenshot 2024-08-01 at 9 44 12 AM

Using very basic test. ie:

 test "test 1" do
    assert false
  end

My environment

FWIW, this is the result on a handful of projects which are all up-to-date with Rails 7 using minitest. I'm using asdf and for this throw away project ruby 3.3.2.

Here are the details for the throwaway rails project:

Output from VSCode for ruby-lsp:

2024-08-01 10:12:17.800 [info] (vscode-ruby-lsp-tester) Discovered version manager asdf
2024-08-01 10:12:17.801 [info] (vscode-ruby-lsp-tester) Running command: `. /opt/homebrew/opt/asdf/libexec/asdf.sh && asdf exec ruby -W0 -rjson -e 'STDERR.print({env: ENV.to_h,yjit:!!defined?(RubyVM::YJIT),version:RUBY_VERSION}.to_json)'` in /Users/andy/Developer/vscode-ruby-lsp-tester using shell: /bin/zsh
2024-08-01 10:12:18.150 [info] (vscode-ruby-lsp-tester) Ruby LSP> Running bundle install for the custom bundle. This may take a while...
Ruby LSP> Command: (bundle check || bundle install) 1>&2

2024-08-01 10:12:18.270 [info] (vscode-ruby-lsp-tester) Bundler can't satisfy your Gemfile's dependencies.
Install missing gems with `bundle install`.

2024-08-01 10:12:18.764 [info] (vscode-ruby-lsp-tester) Fetching gem metadata from https://rubygems.org/
2024-08-01 10:12:18.764 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:18.826 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:18.847 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:18.864 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:18.889 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:18.896 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:18.899 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:18.900 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:18.900 [info] (vscode-ruby-lsp-tester) .
2024-08-01 10:12:19.299 [info] (vscode-ruby-lsp-tester) 
2024-08-01 10:12:19.323 [info] (vscode-ruby-lsp-tester) Resolving dependencies...
2024-08-01 10:12:19.683 [info] (vscode-ruby-lsp-tester) Fetching sorbet-runtime 0.5.11504
2024-08-01 10:12:19.869 [info] (vscode-ruby-lsp-tester) Installing sorbet-runtime 0.5.11504
2024-08-01 10:12:19.882 [info] (vscode-ruby-lsp-tester) Fetching ruby-lsp-rails 0.3.11
2024-08-01 10:12:19.900 [info] (vscode-ruby-lsp-tester) Installing ruby-lsp-rails 0.3.11
2024-08-01 10:12:27.695 [info] (vscode-ruby-lsp-tester) Bundle complete! 17 Gemfile dependencies, 84 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
2024-08-01 10:12:28.158 [info] (vscode-ruby-lsp-tester) Initializing Ruby LSP v0.17.10...
2024-08-01 10:12:28.180 [info] (vscode-ruby-lsp-tester) Finished initializing Ruby LSP!
2024-08-01 10:12:28.209 [info] (vscode-ruby-lsp-tester) Activating Ruby LSP Rails addon v0.3.11
2024-08-01 10:12:28.212 [info] (vscode-ruby-lsp-tester) Ruby LSP Rails booting server
2024-08-01 10:12:32.165 [info] (vscode-ruby-lsp-tester) Finished booting Ruby LSP Rails server

Gem list for lsp:

ruby-lsp (0.17.10)
ruby-lsp-rails (0.3.11)

Bundle list (just in case you want more details and a longer description :-) ):

➜  vscode-ruby-lsp-tester git:(main) ✗ bundle update rails
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using rack 3.1.7 (was 2.2.9)
Using activesupport 7.1.3.4 (was 7.0.8.4)
Using activemodel 7.1.3.4 (was 7.0.8.4)
Using actionview 7.1.3.4 (was 7.0.8.4)
Using activerecord 7.1.3.4 (was 7.0.8.4)
Using actionpack 7.1.3.4 (was 7.0.8.4)
Using activejob 7.1.3.4 (was 7.0.8.4)
Using actioncable 7.1.3.4 (was 7.0.8.4)
Using activestorage 7.1.3.4 (was 7.0.8.4)
Using actionmailer 7.1.3.4 (was 7.0.8.4)
Using railties 7.1.3.4 (was 7.0.8.4)
Using actionmailbox 7.1.3.4 (was 7.0.8.4)
Using actiontext 7.1.3.4 (was 7.0.8.4)
Using rails 7.1.3.4 (was 7.0.8.4)
Bundle updated!
➜  vscode-ruby-lsp-tester git:(main) ✗ bundle list
Gems included by the bundle:
  * actioncable (7.1.3.4)
  * actionmailbox (7.1.3.4)
  * actionmailer (7.1.3.4)
  * actionpack (7.1.3.4)
  * actiontext (7.1.3.4)
  * actionview (7.1.3.4)
  * activejob (7.1.3.4)
  * activemodel (7.1.3.4)
  * activerecord (7.1.3.4)
  * activestorage (7.1.3.4)
  * activesupport (7.1.3.4)
  * addressable (2.8.7)
  * base64 (0.2.0)
  * bigdecimal (3.1.8)
  * bindex (0.8.1)
  * bootsnap (1.18.3)
  * builder (3.3.0)
  * capybara (3.40.0)
  * concurrent-ruby (1.3.3)
  * connection_pool (2.4.1)
  * crass (1.0.6)
  * date (3.3.4)
  * debug (1.9.2)
  * drb (2.2.1)
  * erubi (1.13.0)
  * globalid (1.2.1)
  * i18n (1.14.5)
  * importmap-rails (2.0.1)
  * io-console (0.7.2)
  * irb (1.14.0)
  * jbuilder (2.12.0)
  * logger (1.6.0)
  * loofah (2.22.0)
  * mail (2.8.1)
  * marcel (1.0.4)
  * matrix (0.4.2)
  * mini_mime (1.1.5)
  * minitest (5.24.1)
  * msgpack (1.7.2)
  * mutex_m (0.2.0)
  * net-imap (0.4.14)
  * net-pop (0.1.2)
  * net-protocol (0.2.2)
  * net-smtp (0.5.0)
  * nio4r (2.7.3)
  * nokogiri (1.16.7)
  * psych (5.1.2)
  * public_suffix (6.0.1)
  * puma (5.6.8)
  * racc (1.8.1)
  * rack (3.1.7)
  * rack-session (2.0.0)
  * rack-test (2.1.0)
  * rackup (2.1.0)
  * rails (7.1.3.4)
  * rails-dom-testing (2.2.0)
  * rails-html-sanitizer (1.6.0)
  * railties (7.1.3.4)
  * rake (13.2.1)
  * rdoc (6.7.0)
  * redis (4.8.1)
  * regexp_parser (2.9.2)
  * reline (0.5.9)
  * rexml (3.3.4)
  * rubyzip (2.3.2)
  * selenium-webdriver (4.23.0)
  * sprockets (4.2.1)
  * sprockets-rails (3.5.2)
  * sqlite3 (1.7.3)
  * stimulus-rails (1.3.3)
  * stringio (3.1.1)
  * strscan (3.1.0)
  * thor (1.3.1)
  * timeout (0.4.1)
  * turbo-rails (2.0.6)
  * tzinfo (2.0.6)
  * web-console (4.2.1)
  * webrick (1.8.1)
  * websocket (1.2.11)
  * websocket-driver (0.7.6)
  * websocket-extensions (0.1.5)
  * xpath (3.2.0)
  * zeitwerk (2.6.17)
Use `bundle info` to print more detailed information about a gem
andyw8 commented 1 month ago

Hi @andypeters, this is a known limitation, there is some work happening here to try improve it:

https://github.com/Shopify/ruby-lsp/pull/2230

(I moved the issue to ruby-lsp-rails, but I'm going to move it back since this is a more general problem)

andypeters commented 1 month ago

Hey thanks @andyw8! Dang, I missed looking at active pull requests. My instinct told me you all knew about it, but you never know. All good and happy to hear you all are trying to get it working. I'll subscribe to that pull request and follow along.