Not working as expected with minitest reporting

[mikevoets]

Overview

I am using shoulda with minitest-reporters to generate JUnit reports for my test suite.

The JUnitReporter class from the minitest-reporters gem does not work as expected with the shoulda gem. Initially I thought this would be an issue in the reporters gem, but the maintainer of that gem pointed out it's more likely an issue in shoulda-context. This is what the maintainer said in https://github.com/minitest-reporters/minitest-reporters/issues/344:

I do believe this may need to be fixed in shoulda-context. Our JUnitReporter gets the file path from the Minitest::Result#source_location (here is where that gets set). It appears this attribute ends up recording a location in shoulda-context when using should with a matcher as the argument. Ideally shoulda-context could find a way to record the source location from the actual test class instead.

Here are the reproduction steps:

# frozen_string_literal: true
# test_report.rb

require 'bundler/inline'

gemfile(true) do
  ruby '3.0.6'
  source 'https://rubygems.org'
  gem 'rails', '7.0.6'
  gem 'minitest', '5.18.0'
  gem 'minitest-reporters', '1.6.0'
  gem 'shoulda', '4.0.0'
end

require 'active_record'
require 'action_controller'
require 'minitest/autorun'
require 'shoulda'

Shoulda::Matchers.configure do |config|
  config.integrate do |with|
    with.test_framework :minitest
    with.library :rails
  end
end

Minitest::Reporters.use!([ Minitest::Reporters::JUnitReporter.new ])

class User < ActiveRecord::Base
  validates :email, presence: true
end

class UserTest < ActiveSupport::TestCase
  should validate_presence_of(:email)
end

Expected behaviour

It generates a test/reports/TEST-UserTest.xml with file="test_report.rb" as an attribute key-value pair in the testcase tag.

Actual behaviour

The file attribute has the value of ../../.rbenv/versions/3.0.6/lib/ruby/gems/3.0.0/gems/shoulda-context-2.0.0/lib/shoulda/context/context.rb.

Notes

• If you replace UserTest with the following it works as expected:

  class UserTest < ActiveSupport::TestCase
  should 'be true' do
  assert true
  end
  end