Y24-131 - Fix setting up the integration suite to work with Sequencescape running with Rails 6.1 defaults

[dasunpubudumal]

Describe the Housekeeping

When setting up Limber for running the integration suite along with sequencescape configured with 6.1 defaults in https://github.com/sanger/sequencescape/issues/4005, we encountered an exception.

The command that prompted this exception was:

bundle exec rake config:generate

in Limber.

The purpose of this story is to fix the issue in sequenscescape that prevents Limber's setup.

Blocking issues Describe any other issues or tickets that may be blocking this change.

Additional context

Use the following steps to run the integration suite locally.

You need four terminal windows:

1. Sequencescape Terminal
2. Jobs Terminal
3. Limber Terminal
4. Integration Suite Terminal

Sequencescape Terminal:

bundle exec rake db:reset
bundle exec rake application:post_deploy
bundle exec rails s

Jobs Terminal: Run worker

bundle exec rake jobs:work

Limber Terminal: Run config generate task, and run server

bundle exec rake config:generate
bundle exec rails s

Integration Suite Terminal:

ENVIRONMENT=local bundle exec rspec 

[dasunpubudumal]

Stacktrace of the error (in sequencescape):

actionpack (6.1.7.7) lib/action_controller/metal/request_forgery_protection.rb:211:in `handle_unverified_request'
actionpack (6.1.7.7) lib/action_controller/metal/request_forgery_protection.rb:243:in `handle_unverified_request'
actionpack (6.1.7.7) lib/action_controller/metal/request_forgery_protection.rb:238:in `verify_authenticity_token'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:427:in `block in make_lambda'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:198:in `block (2 levels) in halting'
actionpack (6.1.7.7) lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:199:in `block in halting'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:512:in `block in invoke_before'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:512:in `each'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:512:in `invoke_before'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:115:in `block in run_callbacks'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:137:in `run_callbacks'
actionpack (6.1.7.7) lib/abstract_controller/callbacks.rb:41:in `process_action'
actionpack (6.1.7.7) lib/action_controller/metal/rescue.rb:22:in `process_action'
actionpack (6.1.7.7) lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
activesupport (6.1.7.7) lib/active_support/notifications.rb:203:in `block in instrument'
activesupport (6.1.7.7) lib/active_support/notifications/instrumenter.rb:24:in `instrument'
activesupport (6.1.7.7) lib/active_support/notifications.rb:203:in `instrument'
actionpack (6.1.7.7) lib/action_controller/metal/instrumentation.rb:33:in `process_action'
actionpack (6.1.7.7) lib/action_controller/metal/params_wrapper.rb:249:in `process_action'
activerecord (6.1.7.7) lib/active_record/railties/controller_runtime.rb:27:in `process_action'
actionpack (6.1.7.7) lib/abstract_controller/base.rb:165:in `process'
actionview (6.1.7.7) lib/action_view/rendering.rb:39:in `process'
actionpack (6.1.7.7) lib/action_controller/metal.rb:190:in `dispatch'
actionpack (6.1.7.7) lib/action_controller/metal.rb:254:in `dispatch'
actionpack (6.1.7.7) lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
actionpack (6.1.7.7) lib/action_dispatch/routing/route_set.rb:33:in `serve'
actionpack (6.1.7.7) lib/action_dispatch/journey/router.rb:50:in `block in serve'
actionpack (6.1.7.7) lib/action_dispatch/journey/router.rb:32:in `each'
actionpack (6.1.7.7) lib/action_dispatch/journey/router.rb:32:in `serve'
actionpack (6.1.7.7) lib/action_dispatch/routing/route_set.rb:842:in `call'
flipper (0.25.4) lib/flipper/middleware/memoizer.rb:72:in `memoized_call'
flipper (0.25.4) lib/flipper/middleware/memoizer.rb:37:in `call'
rack (2.2.8.1) lib/rack/tempfile_reaper.rb:15:in `call'
rack (2.2.8.1) lib/rack/etag.rb:27:in `call'
rack (2.2.8.1) lib/rack/conditional_get.rb:40:in `call'
rack (2.2.8.1) lib/rack/head.rb:12:in `call'
actionpack (6.1.7.7) lib/action_dispatch/http/permissions_policy.rb:22:in `call'
actionpack (6.1.7.7) lib/action_dispatch/http/content_security_policy.rb:19:in `call'
bullet (7.1.6) lib/bullet/rack.rb:14:in `call'
rack (2.2.8.1) lib/rack/session/abstract/id.rb:266:in `context'
rack (2.2.8.1) lib/rack/session/abstract/id.rb:260:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/cookies.rb:697:in `call'
activerecord (6.1.7.7) lib/active_record/migration.rb:601:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
activesupport (6.1.7.7) lib/active_support/callbacks.rb:98:in `run_callbacks'
actionpack (6.1.7.7) lib/action_dispatch/middleware/callbacks.rb:26:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
railties (6.1.7.7) lib/rails/rack/logger.rb:37:in `call_app'
railties (6.1.7.7) lib/rails/rack/logger.rb:28:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/request_id.rb:26:in `call'
rack (2.2.8.1) lib/rack/method_override.rb:24:in `call'
rack (2.2.8.1) lib/rack/runtime.rb:22:in `call'
activesupport (6.1.7.7) lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/executor.rb:14:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/static.rb:24:in `call'
rack (2.2.8.1) lib/rack/sendfile.rb:110:in `call'
actionpack (6.1.7.7) lib/action_dispatch/middleware/host_authorization.rb:148:in `call'
rack-cors (2.0.2) lib/rack/cors.rb:102:in `call'
rack-mini-profiler (3.3.1) lib/mini_profiler.rb:334:in `call'
vite_ruby (3.5.0) lib/vite_ruby/dev_server_proxy.rb:22:in `perform_request'
rack-proxy (0.7.7) lib/rack/proxy.rb:87:in `call'
railties (6.1.7.7) lib/rails/engine.rb:539:in `call'
puma (6.4.2) lib/puma/configuration.rb:272:in `call'
puma (6.4.2) lib/puma/request.rb:100:in `block in handle_request'
puma (6.4.2) lib/puma/thread_pool.rb:378:in `with_force_shutdown'
puma (6.4.2) lib/puma/request.rb:99:in `handle_request'
puma (6.4.2) lib/puma/server.rb:464:in `process_client'
puma (6.4.2) lib/puma/server.rb:245:in `block in run'
puma (6.4.2) lib/puma/thread_pool.rb:155:in `block in spawn_thread'

[andrewsparkes]

I did a fresh install of Dasun's 6.1 Sequencescape branch (https://github.com/sanger/sequencescape/tree/4005-dpl-936-3-sequencescape-update-rails-config-defaults-to-version-61). I had to use a more up to date node (v21) as I got an install error with v18 (Dasun has v20) I was able to start the Sequencescape server and job worker. I was able to run config:generate in Limber without any error. I was able to run Sequencescape rspec and just had 2 failed tests (the expected NPG related ones):

rspec './spec/controllers/npg_actions/assets_controller_spec.rb[1:1:4:1:1]' # NpgActions::AssetsController#pass with both an active and an additional cancelled request behaves like a passed state change renders and creates events
rspec './spec/controllers/npg_actions/assets_controller_spec.rb[1:2:2:1:1]' # NpgActions::AssetsController#fail with both a valid and an additional cancelled request behaves like a failed state change renders and creates events

Also quite a few repeated warning messages: sequencescape/lib/tasks/insdc/import_countries.rake:4 (and 11): warning: already initialized constant INSDC_COUNTRIES_DEFAULTS and WARNING: Usingexpect { }.not_to raise_error(SpecificErrorClass, message)risks false positives, since literally any other error would cause the expectation to pass, including those raised by Ruby (e.g.NoMethodError,NameErrorandArgumentError), meaning the code you are intending to test may not even get reached.

I was able to run the main Heron integration suite test (heron_lthr_pipeline_spec.rb, a complex pipeline test) and it passed fine.

So I was not able to replicate the error Dasun saw.

[dasunpubudumal]

It was probably an issue with my local setup. If this runs fine I think we will be able to move forward with https://github.com/sanger/sequencescape/issues/4138.

[andrewsparkes]

Resetting the database and then running config:generate replicated the error on creating plate purposes.

I found that adding this line in /config/environmemts/development.rb prevents the error occurring: config.action_controller.allow_forgery_protection = false

This is not a solution as it would be a security fix, but it indicates the forgery protection is triggering the error.

[andrewsparkes]

Discovered that we can add this line in the v2 api resource controller base class to disable CSFR authentication.

skip_before_action :verify_authenticity_token

This authentication was not switched on for rails 5. Plus we are using api key authentication for security which is still being checked.