2.0 Regression: allow_remote_database_url and url_allowlist not working anymore

tmaier commented 3 years ago

Dependabot just upgraded database_cleaner-active_record to 2.0 and now all tests fail. I used to have DatabaseCleaner.allow_remote_database_url = true in my code.

Open to see error message and trace log

```text Failure/Error: require File.expand_path('../config/environment', __dir__) DatabaseCleaner::Safeguard::Error::RemoteDatabaseUrl: ENV['DATABASE_URL'] is set to a remote URL. Please refer to https://github.com/DatabaseCleaner/database_cleaner#safeguards # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:45:in `run' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `block in run' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `each' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `run' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaner.rb:31:in `initialize' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `new' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `add_cleaner' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `block in []' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `fetch' # /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `[]' # /usr/local/bundle/gems/database_cleaner-active_record-2.0.0/lib/database_cleaner/active_record.rb:7:in `

' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require' # /usr/local/bundle/gems/database_cleaner-active_record-2.0.0/lib/database_cleaner-active_record.rb:1:in `

' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require' # /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:174:in `require' # ./config/application.rb:22:in `

' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:53:in `require_relative' # ./config/environment.rb:4:in `

' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require' # ./spec/rails_helper.rb:6:in `

' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require' # ./spec/uploaders/invoice_uploader_spec.rb:3:in `

' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load' # /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:318:in `block in load' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency' # /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:318:in `load' ```

I also tried to set DatabaseCleaner.url_allowlist = ['postgres://postgres@postgres'], but this did not help.

tmaier commented 3 years ago

Issue opened at wrong repository. Closing it in favour of https://github.com/DatabaseCleaner/database_cleaner-active_record/issues/51

botandrose commented 3 years ago

This is actually the correct place! The safeguard lives in core, so it can be shared by all adapters.

botandrose commented 3 years ago

@tmaier Hmm. The safeguards are still in place, and should be working. I'm guessing its something specific to your usage of database cleaner, plus some refactoring that I did, and together they are having some unintended consequences.

Can you reduce your project down to a minimum reproduction of this issue, and share it with me? If I can reproduce it, I can fix it!

samuelgrigolato commented 3 years ago

I had something similar happening here. The error was happening right at the require line, so the code didn't yet had the time to reach the allow_remote_database_url config. Changing to the env var strategy (i.e. exporting DATABASE_CLEANER_ALLOW_REMOTE_DATABASE_URL=true) worked for me.

samuelgrigolato commented 3 years ago

Here's the stack:

DatabaseCleaner::Safeguard::Error::RemoteDatabaseUrl:
  ENV['DATABASE_URL'] is set to a remote URL. Please refer to https://github.com/DatabaseCleaner/database_cleaner#safeguards
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:45:in `run'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `block in run'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `each'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `run'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaner.rb:31:in `initialize'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `new'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `add_cleaner'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `block in []'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `fetch'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `[]'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-sequel-2.0.0/lib/database_cleaner/sequel.rb:7:in `<top (required)>'
# ./spec/spec_helper.rb:35:in `require'

spec_helper.rb:35 is require "database_cleaner/sequel".

jcoglan commented 3 years ago

We're seeing a similar thing happening when we run assets:precompile, because that causes config/application.rb to be loaded, which calls Bundler.require(*Rails.groups) -- simply loading the gem triggers this exception, even if the process never calls it, so there's no way of configuring the DatabaseCleaner class to turn off this safeguard.

tmaier commented 3 years ago

I tried to configure it right after the Bundler.require statement in the application.rb. This did not fix the problem. So I agree with @jcoglan.

botandrose commented 3 years ago

Okay, opened #673 to attempt to fix this bug. Can you folks try out the branch and let me know if it fixes your issues?

You can do this by changing the database_cleaner line in your Gemfile to the following:

gem "database_cleaner", git: "https://github.com/DatabaseCleaner/database_cleaner", branch: "delay_safeguard"

tmaier commented 3 years ago

Hi @botandrose, I tried it out. It works for me. Thanks mate!‚

botandrose commented 3 years ago

Okay, great! I'll push this out as v2.0.1, presently. Thanks for the detailed reports, detective work, and testing, y'all!

DatabaseCleaner / database_cleaner

2.0 Regression: allow_remote_database_url and url_allowlist not working anymore #671