search

thoughtbot / appraisal

A Ruby library for testing your library against different versions of dependencies.
https://thoughtbot.com
MIT License
1.25k stars 107 forks source link

Appraisal loading wrong versions #196

Closed coding-bunny closed 1 year ago

coding-bunny commented 1 year ago

Hello,

I'm running into something weird with a project setup on our CircleCI. I'm working on a Ruby on Rails Engine, and of course I'd like to test it against various combinations of Ruby and Ruby on Rails. So I've created an Appraisals file that looks like this:

# This is the Appraisals file that determines which specific gems should be tested based upon the selected
# Ruby version and Ruby on Rails framework.
# Because each of these configurations have their own requirements, we will build a compatibility matrix and
# make sure that each step includes the correct version of Ruby on Rails, and freezes the dependencies to the
# requirements of that version.

# Up to Ruby 2.5, we can test any version that's below Rails 6
if ::Gem::Version.new(RUBY_VERSION) < ::Gem::Version.new('2.5.0')
    appraise 'rails32' do
        gem 'rails', '~> 3.2.0'
    end

    appraise 'rails40' do
        gem 'rails', '~> 4.0.0'
    end

    appraise 'rails41' do
        gem 'rails', '~> 4.1.0'
    end

    appraise 'rails42' do
        gem 'rails', '~> 4.2.0'
    end

    appraise 'rails50' do
        gem 'rails', '~> 5.0.0'
    end
end

if ::Gem::Version.new(RUBY_VERSION) < ::Gem::Version.new('2.6.0')
    appraise 'rails51' do
        gem 'rails', '~> 5.1.0'
    end
end

if ::Gem::Version.new(RUBY_VERSION) < ::Gem::Version.new('2.7.0')
    appraise 'rails52' do
        gem 'rails', '~> 5.2.0'
    end
end

# For Ruby versions >= 2.5.0 and < 3.0 we can test all Rails 6
if ::Gem::Version.new(RUBY_VERSION) >= ::Gem::Version.new('2.5.0') && ::Gem::Version.new(RUBY_VERSION) < ::Gem::Version.new('3.0.0')
    appraise 'rails60' do
        gem 'rails', '~> 6.0.0'
    end

    appraise 'rails61' do
        gem 'rails', '~> 6.1.0'
    end
end

# Ruby on Rails 7.0.0 recommends Ruby to be >= 2.7.0
if ::Gem::Version.new(RUBY_VERSION) >= ::Gem::Version.new('2.7.0')
    appraise 'rails70' do
        gem 'rails', '7.0.0'
    end
end

# Ruby on Rails 7.0.x recommend Ruby to be >= 3.1.0
if ::Gem::Version.new(RUBY_VERSION) >= ::Gem::Version.new('3.1.0')
    appraise 'rails7' do
        gem 'rails', '~> 7.0.0'
    end
end

Based on the Ruby version that's being used, it generates the correct Gemfiles, and they seem to have the correct entries in them. For example on Ruby 2.2, when looking at the file generated for Ruby on Rails 3.2, I get the following:

# gemfiles/rails32.gemfile
# This file was generated by Appraisal

source "https://rubygems.org"

gem "test-unit", "~> 3.0"
gem "rails", "~> 3.2.0"

group :test do
  gem "appraisal"
  gem "minitest-ci", require: false
  gem "simplecov", require: false
  gem "jquery-rails"
end

gemspec path: "../"

And the corresponding lockfile looks correct as well. However when I run the tests on the CI system, everything fails because the versions don't match. This is the output for example for the Ruby on Rails 3.2 tests using Ruby 2.2:

== Running Appraisal Install ==
>> bundle check --gemfile='/root/project/gemfiles/rails32.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails32.gemfile' --retry 1
The Gemfile's dependencies are satisfied
>> bundle check --gemfile='/root/project/gemfiles/rails40.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails40.gemfile' --retry 1
The Gemfile's dependencies are satisfied
>> bundle check --gemfile='/root/project/gemfiles/rails41.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails41.gemfile' --retry 1
The Gemfile's dependencies are satisfied
>> bundle check --gemfile='/root/project/gemfiles/rails42.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails42.gemfile' --retry 1
The Gemfile's dependencies are satisfied
>> bundle check --gemfile='/root/project/gemfiles/rails50.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails50.gemfile' --retry 1
The Gemfile's dependencies are satisfied
>> bundle check --gemfile='/root/project/gemfiles/rails51.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails51.gemfile' --retry 1
The Gemfile's dependencies are satisfied
>> bundle check --gemfile='/root/project/gemfiles/rails52.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails52.gemfile' --retry 1
The Gemfile's dependencies are satisfied
== Running tests ==
>> BUNDLE_GEMFILE=/root/project/gemfiles/rails32.gemfile bundle exec rake test
/root/project/vendor/bundle/ruby/2.2.0/gems/tzinfo-1.2.10/lib/tzinfo/zoneinfo_timezone_info.rb:506: warning: shadowing outer local variable - i
/root/project/vendor/bundle/ruby/2.2.0/gems/rack-2.1.4.1/lib/rack/media_type.rb:18: warning: `&' interpreted as argument prefix
DEPRECATION WARNING: `secrets.secret_token` is deprecated in favor of `secret_key_base` and will be removed in Rails 6.0. (called from <top (required)> at /root/project/test/dummy/config/environment.rb:5)
/root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/dynamic_matchers.rb:22:in `method_missing': undefined method `whitelist_attributes=' for #<Class:0x007ff907ccf5d0> (NoMethodError)
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/railtie.rb:124:in `block (3 levels) in <class:Railtie>'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/railtie.rb:123:in `each'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/railtie.rb:123:in `block (2 levels) in <class:Railtie>'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:71:in `instance_eval'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:71:in `block in execute_hook'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:62:in `with_execution_control'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:67:in `execute_hook'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:52:in `block in run_load_hooks'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:51:in `each'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:51:in `run_load_hooks'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/base.rb:328:in `<module:ActiveRecord>'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/base.rb:27:in `<top (required)>'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/migration.rb:598:in `maintain_test_schema!'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/railties-5.2.8.1/lib/rails/test_help.rb:17:in `<top (required)>'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/dependencies.rb:291:in `require'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/dependencies.rb:291:in `block in require'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/dependencies.rb:257:in `load_dependency'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/dependencies.rb:291:in `require'
        from /root/project/test/test_helper.rb:5:in `<top (required)>'
        from /root/project/test/integration/navigation_test.rb:1:in `require'
        from /root/project/test/integration/navigation_test.rb:1:in `<top (required)>'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb:21:in `require'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb:21:in `block in <main>'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb:6:in `select'
        from /root/project/vendor/bundle/ruby/2.2.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb:6:in `<main>'
rake aborted!
Command failed with status (1)
/root/project/vendor/bundle/ruby/2.2.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:30:in `block in <main>'
/usr/local/bin/bundle:22:in `<main>'
Tasks: TOP => test
(See full trace by running task with --trace)

So despite the Gemfile having the correct entries, Appraisal is loading Ruby on Rails 5.2 instead of 3.2 The bin/test does nothing more than this:

#!/usr/bin/env sh
set -e

echo "== Running Appraisal Install =="

bundle exec appraisal install

echo "== Running tests =="

bundle exec appraisal rake test
coding-bunny commented 1 year ago

running the exact same commands/script locally on my system all work. Switching rubies etc all produces the output I expect, but not on CircleCI...

coding-bunny commented 1 year ago

Copied the output from the last run where I tried to just run all tests per ruby version:

bundle config --local path vendor/bundle
+ bundle config --local path vendor/bundle
You are replacing the current local value of path, which is currently nil
bundle install
+ bundle install
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies....
Fetching rake 13.0.6
Installing rake 13.0.6
Fetching concurrent-ruby 1.1.10
Installing concurrent-ruby 1.1.10
Fetching i18n 1.5.1
Installing i18n 1.5.1
Fetching minitest 5.15.0
Installing minitest 5.15.0
Fetching thread_safe 0.3.6
Installing thread_safe 0.3.6
Fetching tzinfo 1.2.10
Installing tzinfo 1.2.10
Fetching activesupport 5.2.8.1
Installing activesupport 5.2.8.1
Fetching builder 3.2.4
Installing builder 3.2.4
Fetching erubi 1.11.0
Installing erubi 1.11.0
Fetching mini_portile2 2.4.0
Installing mini_portile2 2.4.0
Fetching nokogiri 1.9.1
Installing nokogiri 1.9.1 with native extensions
Fetching rails-dom-testing 2.0.3
Installing rails-dom-testing 2.0.3
Fetching crass 1.0.6
Installing crass 1.0.6
Fetching loofah 2.19.0
Installing loofah 2.19.0
Fetching rails-html-sanitizer 1.4.3
Installing rails-html-sanitizer 1.4.3
Fetching actionview 5.2.8.1
Installing actionview 5.2.8.1
Fetching rack 2.1.4.1
Installing rack 2.1.4.1
Fetching rack-test 2.0.2
Installing rack-test 2.0.2
Fetching actionpack 5.2.8.1
Installing actionpack 5.2.8.1
Fetching nio4r 2.3.1
Installing nio4r 2.3.1 with native extensions
Fetching websocket-extensions 0.1.5
Installing websocket-extensions 0.1.5
Fetching websocket-driver 0.7.5
Installing websocket-driver 0.7.5 with native extensions
Fetching actioncable 5.2.8.1
Installing actioncable 5.2.8.1
Fetching globalid 0.4.2
Installing globalid 0.4.2
Fetching activejob 5.2.8.1
Installing activejob 5.2.8.1
Fetching mini_mime 1.1.2
Installing mini_mime 1.1.2
Fetching mail 2.7.1
Installing mail 2.7.1
Fetching actionmailer 5.2.8.1
Installing actionmailer 5.2.8.1
Fetching activemodel 5.2.8.1
Installing activemodel 5.2.8.1
Fetching arel 9.0.0
Installing arel 9.0.0
Fetching activerecord 5.2.8.1
Installing activerecord 5.2.8.1
Fetching marcel 1.0.2
Installing marcel 1.0.2
Fetching activestorage 5.2.8.1
Installing activestorage 5.2.8.1
Using bundler 1.16.6
Fetching thor 1.2.1
Installing thor 1.2.1
Fetching appraisal 2.2.0
Installing appraisal 2.2.0
Fetching docile 1.3.5
Installing docile 1.3.5
Fetching method_source 1.0.0
Installing method_source 1.0.0
Fetching railties 5.2.8.1
Installing railties 5.2.8.1
Fetching sprockets 3.7.2
Installing sprockets 3.7.2
Fetching sprockets-rails 3.2.2
Installing sprockets-rails 3.2.2
Fetching rails 5.2.8.1
Installing rails 5.2.8.1
Using is_it_ready 0.0.1 from source at `.`
Fetching jquery-rails 4.5.0
Installing jquery-rails 4.5.0
Fetching json 2.5.1
Installing json 2.5.1 with native extensions
Fetching minitest-ci 3.4.0
Installing minitest-ci 3.4.0
Fetching power_assert 2.0.1
Installing power_assert 2.0.1
Fetching simplecov-html 0.10.2
Installing simplecov-html 0.10.2
Fetching simplecov 0.17.1
Installing simplecov 0.17.1
Fetching sqlite3 1.4.4
Installing sqlite3 1.4.4 with native extensions
Fetching test-unit 3.5.3
Installing test-unit 3.5.3
Bundle complete! 9 Gemfile dependencies, 51 gems now installed.
Bundled gems are installed into `./vendor/bundle`
Post-install message from i18n:

HEADS UP! i18n 1.1 changed fallbacks to exclude default locale.
But that may break your application.

Please check your Rails app for 'config.i18n.fallbacks = true'.
If you're using I18n (>= 1.1.0) and Rails (< 5.2.2), this should be
'config.i18n.fallbacks = [I18n.default_locale]'.
If not, fallbacks will be broken in your app by I18n 1.1.x.

For more info see:
https://github.com/svenfuchs/i18n/releases/tag/v1.1.0

# Do any other automated setup that you need to do here
== Running Appraisal Install ==
>> bundle check --gemfile='/root/project/gemfiles/rails32.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails32.gemfile' --retry 1
Bundler can't satisfy your Gemfile's dependencies.
Install missing gems with `bundle install`.
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies....
Fetching rake 12.3.3
Installing rake 12.3.3
Using concurrent-ruby 1.1.10
Fetching i18n 0.9.5
Installing i18n 0.9.5
Fetching multi_json 1.15.0
Installing multi_json 1.15.0
Fetching activesupport 3.2.22.5
Installing activesupport 3.2.22.5
Fetching builder 3.0.4
Installing builder 3.0.4
Fetching activemodel 3.2.22.5
Installing activemodel 3.2.22.5
Fetching erubis 2.7.0
Installing erubis 2.7.0
Fetching journey 1.0.4
Installing journey 1.0.4
Fetching rack 1.4.7
Installing rack 1.4.7
Fetching rack-cache 1.9.0
Installing rack-cache 1.9.0
Fetching rack-test 0.6.3
Installing rack-test 0.6.3
Fetching hike 1.2.3
Installing hike 1.2.3
Fetching tilt 1.4.1
Installing tilt 1.4.1
Fetching sprockets 2.2.3
Installing sprockets 2.2.3
Fetching actionpack 3.2.22.5
Installing actionpack 3.2.22.5
Fetching mime-types 1.25.1
Installing mime-types 1.25.1
Fetching polyglot 0.3.5
Installing polyglot 0.3.5
Fetching treetop 1.4.15
Installing treetop 1.4.15
Fetching mail 2.5.5
Installing mail 2.5.5
Fetching actionmailer 3.2.22.5
Installing actionmailer 3.2.22.5
Fetching arel 3.0.3
Installing arel 3.0.3
Fetching tzinfo 0.3.61
Installing tzinfo 0.3.61
Fetching activerecord 3.2.22.5
Installing activerecord 3.2.22.5
Fetching activeresource 3.2.22.5
Installing activeresource 3.2.22.5
Using bundler 1.16.6
Using thor 1.2.1
Using appraisal 2.2.0
Using docile 1.3.5
Fetching rack-ssl 1.3.4
Installing rack-ssl 1.3.4
Fetching rdoc 3.9.5
Installing rdoc 3.9.5
Fetching railties 3.2.22.5
Installing railties 3.2.22.5
Fetching rails 3.2.22.5
Installing rails 3.2.22.5
Using is_it_ready 0.0.1 from source at `../`
Fetching jquery-rails 3.1.5
Installing jquery-rails 3.1.5
Using json 2.5.1
Using minitest 5.15.0
Using minitest-ci 3.4.0
Using power_assert 2.0.1
Using simplecov-html 0.10.2
Using simplecov 0.17.1
Using sqlite3 1.4.4
Using test-unit 3.5.3
Bundle complete! 11 Gemfile dependencies, 43 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Post-install message from rdoc:
Depending on your version of ruby, you may need to install ruby rdoc/ri data:

<= 1.8.6 : unsupported
 = 1.8.7 : gem install rdoc-data; rdoc-data --install
 = 1.9.1 : gem install rdoc-data; rdoc-data --install
>= 1.9.2 : nothing to do! Yay!
>> bundle check --gemfile='/root/project/gemfiles/rails40.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails40.gemfile' --retry 1
Bundler can't satisfy your Gemfile's dependencies.
Install missing gems with `bundle install`.
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies.........
Using rake 12.3.3
Using concurrent-ruby 1.1.10
Using i18n 0.9.5
Fetching minitest 4.7.5
Installing minitest 4.7.5
Using multi_json 1.15.0
Using thread_safe 0.3.6
Using tzinfo 0.3.61
Fetching activesupport 4.0.13
Installing activesupport 4.0.13
Fetching builder 3.1.4
Installing builder 3.1.4
Using erubis 2.7.0
Fetching rack 1.5.5
Installing rack 1.5.5
Using rack-test 0.6.3
Fetching actionpack 4.0.13
Installing actionpack 4.0.13
Using mini_mime 1.1.2
Using mail 2.7.1
Fetching actionmailer 4.0.13
Installing actionmailer 4.0.13
Fetching activemodel 4.0.13
Installing activemodel 4.0.13
Fetching activerecord-deprecated_finders 1.0.4
Installing activerecord-deprecated_finders 1.0.4
Fetching arel 4.0.2
Installing arel 4.0.2
Fetching activerecord 4.0.13
Installing activerecord 4.0.13
Using bundler 1.16.6
Using thor 1.2.1
Using appraisal 2.2.0
Using docile 1.3.5
Fetching railties 4.0.13
Installing railties 4.0.13
Using sprockets 3.7.2
Fetching sprockets-rails 2.3.3
Installing sprockets-rails 2.3.3
Fetching rails 4.0.13
Installing rails 4.0.13
Using is_it_ready 0.0.1 from source at `../`
Using jquery-rails 3.1.5
Using json 2.5.1
Fetching minitest-ci 2.4.0
Installing minitest-ci 2.4.0
Using power_assert 2.0.1
Using simplecov-html 0.10.2
Using simplecov 0.17.1
Using sqlite3 1.4.4
Using test-unit 3.5.3
Bundle complete! 11 Gemfile dependencies, 37 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
>> bundle check --gemfile='/root/project/gemfiles/rails41.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails41.gemfile' --retry 1
Bundler can't satisfy your Gemfile's dependencies.
Install missing gems with `bundle install`.
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies....
Using rake 12.3.3
Using concurrent-ruby 1.1.10
Using i18n 0.9.5
Fetching json 1.8.6
Installing json 1.8.6 with native extensions
Using minitest 5.15.0
Using thread_safe 0.3.6
Using tzinfo 1.2.10
Fetching activesupport 4.1.16
Installing activesupport 4.1.16
Using builder 3.2.4
Using erubis 2.7.0
Fetching actionview 4.1.16
Installing actionview 4.1.16
Using rack 1.5.5
Using rack-test 0.6.3
Fetching actionpack 4.1.16
Installing actionpack 4.1.16
Using mini_mime 1.1.2
Using mail 2.7.1
Fetching actionmailer 4.1.16
Installing actionmailer 4.1.16
Fetching activemodel 4.1.16
Installing activemodel 4.1.16
Fetching arel 5.0.1.20140414130214
Installing arel 5.0.1.20140414130214
Fetching activerecord 4.1.16
Installing activerecord 4.1.16
Using bundler 1.16.6
Using thor 1.2.1
Using appraisal 2.2.0
Using docile 1.3.5
Fetching railties 4.1.16
Installing railties 4.1.16
Using sprockets 3.7.2
Using sprockets-rails 2.3.3
Fetching rails 4.1.16
Installing rails 4.1.16
Using is_it_ready 0.0.1 from source at `../`
Using jquery-rails 3.1.5
Using minitest-ci 3.4.0
Using power_assert 2.0.1
Using simplecov-html 0.10.2
Using simplecov 0.17.1
Using sqlite3 1.4.4
Using test-unit 3.5.3
Bundle complete! 11 Gemfile dependencies, 36 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
>> bundle check --gemfile='/root/project/gemfiles/rails42.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails42.gemfile' --retry 1
Bundler can't satisfy your Gemfile's dependencies.
Install missing gems with `bundle install`.
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies....
Using rake 12.3.3
Using concurrent-ruby 1.1.10
Using i18n 0.9.5
Using minitest 5.15.0
Using thread_safe 0.3.6
Using tzinfo 1.2.10
Fetching activesupport 4.2.11.3
Installing activesupport 4.2.11.3
Using builder 3.2.4
Using erubis 2.7.0
Using mini_portile2 2.4.0
Using nokogiri 1.9.1
Fetching rails-deprecated_sanitizer 1.0.4
Installing rails-deprecated_sanitizer 1.0.4
Fetching rails-dom-testing 1.0.9
Installing rails-dom-testing 1.0.9
Using crass 1.0.6
Using loofah 2.19.0
Using rails-html-sanitizer 1.4.3
Fetching actionview 4.2.11.3
Installing actionview 4.2.11.3
Fetching rack 1.6.13
Installing rack 1.6.13
Using rack-test 0.6.3
Fetching actionpack 4.2.11.3
Installing actionpack 4.2.11.3
Using globalid 0.4.2
Fetching activejob 4.2.11.3
Installing activejob 4.2.11.3
Using mini_mime 1.1.2
Using mail 2.7.1
Fetching actionmailer 4.2.11.3
Installing actionmailer 4.2.11.3
Fetching activemodel 4.2.11.3
Installing activemodel 4.2.11.3
Fetching arel 6.0.4
Installing arel 6.0.4
Fetching activerecord 4.2.11.3
Installing activerecord 4.2.11.3
Using bundler 1.16.6
Using thor 1.2.1
Using appraisal 2.2.0
Using docile 1.3.5
Fetching railties 4.2.11.3
Installing railties 4.2.11.3
Using sprockets 3.7.2
Using sprockets-rails 3.2.2
Fetching rails 4.2.11.3
Installing rails 4.2.11.3
Using is_it_ready 0.0.1 from source at `../`
Using jquery-rails 4.5.0
Using json 2.5.1
Using minitest-ci 3.4.0
Using power_assert 2.0.1
Using simplecov-html 0.10.2
Using simplecov 0.17.1
Using sqlite3 1.4.4
Using test-unit 3.5.3
Bundle complete! 11 Gemfile dependencies, 45 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
>> bundle check --gemfile='/root/project/gemfiles/rails50.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails50.gemfile' --retry 1
Bundler can't satisfy your Gemfile's dependencies.
Install missing gems with `bundle install`.
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies....
Using rake 12.3.3
Using concurrent-ruby 1.1.10
Using i18n 1.5.1
Using minitest 5.15.0
Using thread_safe 0.3.6
Using tzinfo 1.2.10
Fetching activesupport 5.0.7.2
Installing activesupport 5.0.7.2
Using builder 3.2.4
Using erubis 2.7.0
Using mini_portile2 2.4.0
Using nokogiri 1.9.1
Using rails-dom-testing 2.0.3
Using crass 1.0.6
Using loofah 2.19.0
Using rails-html-sanitizer 1.4.3
Fetching actionview 5.0.7.2
Installing actionview 5.0.7.2
Using rack 2.1.4.1
Using rack-test 0.6.3
Fetching actionpack 5.0.7.2
Installing actionpack 5.0.7.2
Using nio4r 2.3.1
Using websocket-extensions 0.1.5
Fetching websocket-driver 0.6.5
Installing websocket-driver 0.6.5 with native extensions
Fetching actioncable 5.0.7.2
Installing actioncable 5.0.7.2
Using globalid 0.4.2
Fetching activejob 5.0.7.2
Installing activejob 5.0.7.2
Using mini_mime 1.1.2
Using mail 2.7.1
Fetching actionmailer 5.0.7.2
Installing actionmailer 5.0.7.2
Fetching activemodel 5.0.7.2
Installing activemodel 5.0.7.2
Fetching arel 7.1.4
Installing arel 7.1.4
Fetching activerecord 5.0.7.2
Installing activerecord 5.0.7.2
Using bundler 1.16.6
Using thor 1.2.1
Using appraisal 2.2.0
Using docile 1.3.5
Using method_source 1.0.0
Fetching railties 5.0.7.2
Installing railties 5.0.7.2
Using sprockets 3.7.2
Using sprockets-rails 3.2.2
Fetching rails 5.0.7.2
Installing rails 5.0.7.2
Using is_it_ready 0.0.1 from source at `../`
Using jquery-rails 4.5.0
Using json 2.5.1
Using minitest-ci 3.4.0
Using power_assert 2.0.1
Using simplecov-html 0.10.2
Using simplecov 0.17.1
Using sqlite3 1.4.4
Using test-unit 3.5.3
Bundle complete! 11 Gemfile dependencies, 49 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
>> bundle check --gemfile='/root/project/gemfiles/rails51.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails51.gemfile' --retry 1
Bundler can't satisfy your Gemfile's dependencies.
Install missing gems with `bundle install`.
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies....
Using rake 12.3.3
Using concurrent-ruby 1.1.10
Using i18n 1.5.1
Using minitest 5.15.0
Using thread_safe 0.3.6
Using tzinfo 1.2.10
Fetching activesupport 5.1.7
Installing activesupport 5.1.7
Using builder 3.2.4
Using erubi 1.11.0
Using mini_portile2 2.4.0
Using nokogiri 1.9.1
Using rails-dom-testing 2.0.3
Using crass 1.0.6
Using loofah 2.19.0
Using rails-html-sanitizer 1.4.3
Fetching actionview 5.1.7
Installing actionview 5.1.7
Using rack 2.1.4.1
Using rack-test 2.0.2
Fetching actionpack 5.1.7
Installing actionpack 5.1.7
Using nio4r 2.3.1
Using websocket-extensions 0.1.5
Using websocket-driver 0.6.5
Fetching actioncable 5.1.7
Installing actioncable 5.1.7
Using globalid 0.4.2
Fetching activejob 5.1.7
Installing activejob 5.1.7
Using mini_mime 1.1.2
Using mail 2.7.1
Fetching actionmailer 5.1.7
Installing actionmailer 5.1.7
Fetching activemodel 5.1.7
Installing activemodel 5.1.7
Fetching arel 8.0.0
Installing arel 8.0.0
Fetching activerecord 5.1.7
Installing activerecord 5.1.7
Using bundler 1.16.6
Using thor 1.2.1
Using appraisal 2.2.0
Using docile 1.3.5
Using method_source 1.0.0
Fetching railties 5.1.7
Installing railties 5.1.7
Using sprockets 3.7.2
Using sprockets-rails 3.2.2
Fetching rails 5.1.7
Installing rails 5.1.7
Using is_it_ready 0.0.1 from source at `../`
Using jquery-rails 4.5.0
Using json 2.5.1
Using minitest-ci 3.4.0
Using power_assert 2.0.1
Using simplecov-html 0.10.2
Using simplecov 0.17.1
Using sqlite3 1.4.4
Using test-unit 3.5.3
Bundle complete! 11 Gemfile dependencies, 49 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
>> bundle check --gemfile='/root/project/gemfiles/rails52.gemfile' || bundle install --gemfile='/root/project/gemfiles/rails52.gemfile' --retry 1
Resolving dependencies...
The Gemfile's dependencies are satisfied
== Running tests ==
>> BUNDLE_GEMFILE=/root/project/gemfiles/rails32.gemfile bundle exec rake test
/root/project/vendor/bundle/ruby/2.2.0/gems/tzinfo-1.2.10/lib/tzinfo/zoneinfo_timezone_info.rb:506: warning: shadowing outer local variable - i
/root/project/vendor/bundle/ruby/2.2.0/gems/rack-2.1.4.1/lib/rack/media_type.rb:18: warning: `&' interpreted as argument prefix
DEPRECATION WARNING: `secrets.secret_token` is deprecated in favor of `secret_key_base` and will be removed in Rails 6.0. (called from <top (required)> at /root/project/test/dummy/config/environment.rb:5)
/root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/dynamic_matchers.rb:22:in `method_missing': undefined method `whitelist_attributes=' for #<Class:0x007fe397fcf260> (NoMethodError)
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/railtie.rb:124:in `block (3 levels) in <class:Railtie>'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/railtie.rb:123:in `each'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/railtie.rb:123:in `block (2 levels) in <class:Railtie>'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:71:in `instance_eval'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:71:in `block in execute_hook'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:62:in `with_execution_control'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:67:in `execute_hook'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:52:in `block in run_load_hooks'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:51:in `each'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/lazy_load_hooks.rb:51:in `run_load_hooks'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/base.rb:328:in `<module:ActiveRecord>'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/base.rb:27:in `<top (required)>'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activerecord-5.2.8.1/lib/active_record/migration.rb:598:in `maintain_test_schema!'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/railties-5.2.8.1/lib/rails/test_help.rb:17:in `<top (required)>'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/dependencies.rb:291:in `require'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/dependencies.rb:291:in `block in require'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/dependencies.rb:257:in `load_dependency'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/activesupport-5.2.8.1/lib/active_support/dependencies.rb:291:in `require'
    from /root/project/test/test_helper.rb:5:in `<top (required)>'
    from /root/project/test/integration/navigation_test.rb:1:in `require'
    from /root/project/test/integration/navigation_test.rb:1:in `<top (required)>'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/rake-12.3.3/lib/rake/rake_test_loader.rb:17:in `require'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/rake-12.3.3/lib/rake/rake_test_loader.rb:17:in `block in <main>'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/rake-12.3.3/lib/rake/rake_test_loader.rb:5:in `select'
    from /root/project/vendor/bundle/ruby/2.2.0/gems/rake-12.3.3/lib/rake/rake_test_loader.rb:5:in `<main>'
rake aborted!
Command failed with status (1)
/root/project/vendor/bundle/ruby/2.2.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:30:in `block in <main>'
/usr/local/bin/bundle:22:in `<main>'
Tasks: TOP => test
(See full trace by running task with --trace)

Exited with code exit status 1

CircleCI received exit code 1

You can see that Appraisal properly install the correct gem versions for each Gemfile. But when running the tests it mixes them up for some weird reason.

coding-bunny commented 1 year ago

Appraisal works, but it just threw out weird reporting due underlying issues :/