connorshea / vscode-ruby-test-adapter

A Ruby test adapter extension for the VS Code Test Explorer
https://marketplace.visualstudio.com/items?itemName=connorshea.vscode-ruby-test-adapter
MIT License
83 stars 50 forks source link

Tests generated by fresh Rails 6.1.4 app using scaffolding do not return any results #81

Closed DanielJackson-Oslo closed 3 years ago

DanielJackson-Oslo commented 3 years ago

Your environment

Expected behavior

When running tests scaffolded by Rails, the results of the tests show up in green or red, both when running Rails and running a simpler Sinatra app.

Actual behavior

When running the tests for a Rails app, the results never show. Instead the tests remain as they were before running. Everything works as expected in the simpler Sinatra app.

127498818-9f0686a2-9170-470c-bed8-0ae9f1a4cc74

Setting the debug output flag, this is the output I get:

[2021-07-29 13:16:06.226] [INFO] Running Ruby tests ["root"]
[2021-07-29 13:16:06.228] [INFO] Running full test suite.
[2021-07-29 13:16:06.228] [INFO] Running command: bundle exec rake -R $EXT_DIR vscode:minitest:run
[2021-07-29 13:16:07.192] [DEBUG] [CHILD PROCESS OUTPUT] START_OF_TEST_JSON{"version":"5.14.4","summary":{"duration":0.28,"example_count":0,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 0.28, Runs: 0, Assertions: 0, Failures: 0, Errors: 0, Skips: 0","examples":[]}END_OF_TEST_JSON
[2021-07-29 13:16:07.192] [DEBUG] Parsing the below JSON:
[2021-07-29 13:16:07.192] [DEBUG] {"version":"5.14.4","summary":{"duration":0.28,"example_count":0,"failure_count":0,"pending_count":0,"errors_outside_of_examples_count":0},"summary_line":"Total time: 0.28, Runs: 0, Assertions: 0, Failures: 0, Errors: 0, Skips: 0","examples":[]}
[2021-07-29 13:16:07.202] [INFO] Child process has exited. Sending test run finish event.

When I try running bundle exec rake -R $EXT_DIR vscode:minitest:run in terminal, the tests exit correctly, with the following output:

rails_hello_world_app git:(main) ✗ bundle exec rake -R $EXT_DIR vscode:minitest:run
Run options: --seed 29657

# Running:

.......

Finished in 0.323463s, 21.6408 runs/s, 27.8239 assertions/s.
7 runs, 9 assertions, 0 failures, 0 errors, 0 skips

It seems to me like the JSON results are either parsed incorrectly, or passed incorrectly.

Steps to reproduce

1) Set up Rails (See Gemfile below) rails new test, bundle install, etc. 2) Scaffold a primitive model with rails g scaffold Test test:string and migrate the db with rails db:migrate 3) Run the tests (I had to restart vscode to get the tests recognized)

Gemfile:

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '3.0.2'

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails', branch: 'main'
gem 'rails', '~> 6.1.4'
# Use Puma as the app server
gem 'puma', '~> 5.0'
# Use SCSS for stylesheets
gem 'sass-rails', '>= 6'
# Transpile app-like JavaScript. Read more: https://github.com/rails/webpacker
gem 'webpacker', '~> 5.0'
# Turbolinks makes navigating your web application faster. Read more: https://github.com/turbolinks/turbolinks
gem 'turbolinks', '~> 5'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 2.7'
# Use Redis adapter to run Action Cable in production
# gem 'redis', '~> 4.0'
# Use Active Model has_secure_password
# gem 'bcrypt', '~> 3.1.7'

# Needed for Rails with Ruby 3:
gem 'rexml', '~> 3.2', '>= 3.2.4'

# Use Active Storage variant
# gem 'image_processing', '~> 1.2'

# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.4.4', require: false

group :production do
  gem 'pg', '~> 1.2.3'
end

group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
  # Use sqlite3 as the database for Active Record for testing and dev
  gem 'sqlite3', '~> 1.4'
end

group :development do
  # Access an interactive console on exception pages or by calling 'console' anywhere in the code.
  gem 'web-console', '>= 4.1.0'
  # Display performance information such as SQL time and flame graphs for each request in your browser.
  # Can be configured to work on production as well see: https://github.com/MiniProfiler/rack-mini-profiler/blob/master/README.md
  gem 'rack-mini-profiler', '~> 2.0'
  gem 'listen', '~> 3.3'
  # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
  gem 'spring'
end

group :test do
  # Adds support for Capybara system testing and selenium driver
  gem 'capybara', '>= 3.26'
  gem 'selenium-webdriver'
  # Easy installation and use of web drivers to run system tests with browsers
  gem 'webdrivers'
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
# gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

Steps I've tried that didn't work:

1) Adding minitest to the gem file explicitly. AFAICT it did no difference.

DanielJackson-Oslo commented 3 years ago

Possibly you could reproduce without the scaffolding, using just the tests that come with a fresh Rails installation.

DanielJackson-Oslo commented 3 years ago

Seems to be the same issue as #20 and the temporary fix in https://github.com/connorshea/vscode-ruby-test-adapter/issues/20#issuecomment-657348728 works for me!

Closing this one, and bumping #20 instead.