YaleSTC / reservations

Manage equipment loans & reservations. Who can borrow what, for how long?
yalestc.github.io/reservations
MIT License
139 stars 58 forks source link

DenyMissedReservationsJob crashing? #1740

Closed orenyk closed 5 years ago

orenyk commented 5 years ago

Ruh roh:

I, [2018-07-18T05:12:10.988916 #9896]  INFO -- : [ActiveJob] [DailyTasksJob] [daaa1ec5-5939-4fb8-92d1-1820585895ab] Enqueued DenyMissedRequestsJob (Job ID: 7ab8940d-79e6-4887-abee-eb3ac00e1d4d) to Async(default)
I, [2018-07-18T05:12:10.993771 #9896]  INFO -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d] Performing DenyMissedRequestsJob (Job ID: 7ab8940d-79e6-4887-abee-eb3ac00e1d4d) from Async(default)
D, [2018-07-18T05:12:11.002830 #9896] DEBUG -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d]   ESC[1mESC[35m (1.2ms)ESC[0m  ESC[1mESC[35mSET NAMES utf8,  @@SESSION.sql_mode = CONCAT(CONCAT(@@sql_mode
, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'),  @@SESSION.sql_auto_is_null = 0, @@SESSION.wait_timeout = 2147483ESC[0m
D, [2018-07-18T05:12:11.041510 #9896] DEBUG -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d]   ESC[1mESC[35m (36.8ms)ESC[0m  ESC[1mESC[34mSELECT COUNT(*) FROM `reservations` WHERE (start_date < '2018
-07-18') AND `reservations`.`status` = 0ESC[0m
I, [2018-07-18T05:12:11.041916 #9896]  INFO -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d] Found 1 missed requests reservations.
D, [2018-07-18T05:12:11.074007 #9896] DEBUG -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d]   ESC[1mESC[36mReservation Load (31.8ms)ESC[0m  ESC[1mESC[34mSELECT `reservations`.* FROM `reservations` W
HERE (start_date < '2018-07-18') AND `reservations`.`status` = 0 ORDER BY start_date, due_date, reserver_idESC[0m
I, [2018-07-18T05:12:11.074926 #9896]  INFO -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d] Marking as denied:
 #<Reservation id: 82795, reserver_id: 11898, checkout_handler_id: nil, checkin_handler_id: nil, start_date: "2018-07-16", due_date: "2018-07-20", checked_out: nil, checked_in: nil, created_at: "2018-07-16 15:21:03", updated_at: "
2018-07-16 15:21:03", equipment_model_id: 139, equipment_item_id: nil, notes: "### Requested on July 16, 2018 11:21 by [Tyra Pend...", notes_unsent: false, times_renewed: nil, status: "requested", overdue: false, flags: 3>
D, [2018-07-18T05:12:11.075832 #9896] DEBUG -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d]   ESC[1mESC[35m (0.4ms)ESC[0m  ESC[1mESC[35mBEGINESC[0m
D, [2018-07-18T05:12:11.128548 #9896] DEBUG -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d]   ESC[1mESC[35m (0.2ms)ESC[0m  ESC[1mESC[31mROLLBACKESC[0m
E, [2018-07-18T05:12:11.129202 #9896] ERROR -- : [ActiveJob] [DenyMissedRequestsJob] [7ab8940d-79e6-4887-abee-eb3ac00e1d4d] Error performing DenyMissedRequestsJob (Job ID: 7ab8940d-79e6-4887-abee-eb3ac00e1d4d) from Async(default) 
in 133.72ms: ActiveRecord::StatementInvalid (Mysql2::Error: MySQL client is not connected: ROLLBACK):
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/mysql2-0.4.8/lib/mysql2/client.rb:120:in `_query'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/mysql2-0.4.8/lib/mysql2/client.rb:120:in `block in query'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/mysql2-0.4.8/lib/mysql2/client.rb:119:in `handle_interrupt'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/mysql2-0.4.8/lib/mysql2/client.rb:119:in `query'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:214:in `block (2 levels) in execute'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/dependencies/interlock.rb:46:in `block in permit_concurrent_loads'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/concurrency/share_lock.rb:185:in `yield_shares'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/dependencies/interlock.rb:45:in `permit_concurrent_loads'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:213:in `block in execute'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:612:in `block (2 levels) in log'
/usr/local/rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:611:in `block in log'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:603:in `log'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:212:in `execute'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/mysql/database_statements.rb:22:in `execute'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:240:in `exec_rollback_db_transaction'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:278:in `rollback_db_transaction'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/query_cache.rb:17:in `rollback_db_transaction'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/transaction.rb:135:in `rollback'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/transaction.rb:185:in `block in rollback_transaction'
/usr/local/rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/transaction.rb:183:in `rollback_transaction'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/transaction.rb:204:in `ensure in block in within_new_transaction'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/transaction.rb:206:in `block in within_new_transaction'
/usr/local/rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/transaction.rb:191:in `within_new_transaction'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/connection_adapters/abstract/database_statements.rb:225:in `transaction'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/transactions.rb:210:in `transaction'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/transactions.rb:381:in `with_transaction_returning_status'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/transactions.rb:308:in `block in save'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/transactions.rb:323:in `rollback_active_record_state!'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/transactions.rb:307:in `save'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/suppressor.rb:42:in `save'
/ror_prd/apps/reservations.yale.edu/bmec/releases/20180712160331/app/models/reservation.rb:188:in `expire!'
/ror_prd/apps/reservations.yale.edu/bmec/releases/20180712160331/app/jobs/deny_missed_requests_job.rb:22:in `block in run'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/relation/delegation.rb:39:in `each'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activerecord-5.1.2/lib/active_record/relation/delegation.rb:39:in `each'
/ror_prd/apps/reservations.yale.edu/bmec/releases/20180712160331/app/jobs/deny_missed_requests_job.rb:20:in `run'
/ror_prd/apps/reservations.yale.edu/bmec/releases/20180712160331/app/jobs/reservation_job.rb:9:in `perform'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/execution.rb:37:in `block in perform_now'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:108:in `block in run_callbacks'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/i18n-0.9.5/lib/i18n.rb:268:in `with_locale'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/translation.rb:7:in `block (2 levels) in <module:Translation>'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:117:in `instance_exec'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/logging.rb:24:in `block (4 levels) in <module:Logging>'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/notifications.rb:166:in `block in instrument'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/notifications.rb:166:in `instrument'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/logging.rb:23:in `block (3 levels) in <module:Logging>'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/logging.rb:44:in `block in tag_logger'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/tagged_logging.rb:69:in `block in tagged'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/tagged_logging.rb:26:in `tagged'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/tagged_logging.rb:69:in `tagged'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/logging.rb:44:in `tag_logger'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/logging.rb:20:in `block (2 levels) in <module:Logging>'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:117:in `instance_exec'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:135:in `run_callbacks'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/execution.rb:33:in `perform_now'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/execution.rb:22:in `block in execute'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:108:in `block in run_callbacks'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/railtie.rb:26:in `block (4 levels) in <class:Railtie>'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/execution_wrapper.rb:85:in `wrap'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/reloader.rb:68:in `block in wrap'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/execution_wrapper.rb:85:in `wrap'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/reloader.rb:67:in `wrap'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/railtie.rb:25:in `block (3 levels) in <class:Railtie>'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:117:in `instance_exec'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activesupport-5.1.2/lib/active_support/callbacks.rb:135:in `run_callbacks'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/execution.rb:20:in `execute'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/activejob-5.1.2/lib/active_job/queue_adapters/async_adapter.rb:68:in `perform'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/ror_prd/apps/reservations.yale.edu/bmec/shared/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'

Looks like part of the job is working but not all of it 😞.

orenyk commented 5 years ago

Also fixed via #1746