search

bensheldon / good_job

Multithreaded, Postgres-based, Active Job backend for Ruby on Rails.
https://goodjob-demo.herokuapp.com/
MIT License
2.69k stars 200 forks source link

2.99 -> 3.0.2 migration not applying cleanly from a fresh DB #1435

Closed mogul closed 3 months ago

mogul commented 3 months ago

I'm attempting to upgrade good_job from 2.99 to 3.99.1, following the incremental update instructions.

Incrementally generating migrations with rails g good_job:update and then running rails db:migrate seems to work fine up through good_job 3.4.8.

However, if I then run rails db:migrate:reset, applying the migrations fails with:

rake aborted!
StandardError: An error has occurred, all later migrations canceled: (StandardError)

PG::UndefinedColumn: ERROR:  column good_jobs.retried_good_job_id does not exist
LINE 1: ...good_jobs"."active_job_id" FROM "good_jobs" WHERE "good_jobs...
                                                             ^

Can you help me figure out where I'm going wrong, or if there's a bug?

--> details, click to expand ```bash bretamogilefsky@FCOH2J-H4G9T32J all_sorns % RAILS_ENV=development bundle exec rails g good_job:update Running via Spring preloader in process 61942 skip db/migrate/20240718004316_create_good_jobs.rb identical db/migrate/20240718000130_create_good_job_settings.rb bretamogilefsky@FCOH2J-H4G9T32J all_sorns % RAILS_ENV=development bundle exec rails db:migrate Running via Spring preloader in process 61980 == 20240718000130 CreateGoodJobSettings: migrating ============================ -- create_table(:good_job_settings, {:id=>:uuid}) -> 0.0131s == 20240718000130 CreateGoodJobSettings: migrated (0.0143s) =================== ### All good here... But what if we try to migrate from scratch? bretamogilefsky@FCOH2J-H4G9T32J all_sorns % RAILS_ENV=development bundle exec rails db:migrate:reset Running via Spring preloader in process 62031 Dropped database 'all_sorns_development' Dropped database 'all_sorns_test' Created database 'all_sorns_development' Created database 'all_sorns_test' == 20200914224855 CreateAgencies: migrating =================================== -- create_table(:agencies) -> 0.0058s == 20200914224855 CreateAgencies: migrated (0.0059s) ========================== == 20200914225155 CreateSorns: migrating ====================================== -- create_table(:sorns) -> 0.0166s == 20200914225155 CreateSorns: migrated (0.0166s) ============================= == 20200914230142 CreateGoodJobs: migrating =================================== -- enable_extension("pgcrypto") -> 0.0228s -- create_table(:good_jobs, {:id=>:uuid}) -> 0.0040s -- add_index(:good_jobs, :scheduled_at, {:where=>"(finished_at IS NULL)"}) -> 0.0031s -- add_index(:good_jobs, [:queue_name, :scheduled_at], {:where=>"(finished_at IS NULL)"}) -> 0.0038s == 20200914230142 CreateGoodJobs: migrated (0.0339s) ========================== == 20200921223917 AddHistoryToSorns: migrating ================================ -- add_column(:sorns, :history, :string) -> 0.0029s == 20200921223917 AddHistoryToSorns: migrated (0.0030s) ======================= == 20200923212702 AddJsonToSorn: migrating ==================================== -- add_column(:sorns, :sections, :json) -> 0.0018s == 20200923212702 AddJsonToSorn: migrated (0.0018s) =========================== == 20200924185945 AddMoreToSorn: migrating ==================================== -- add_column(:sorns, :purpose, :string) -> 0.0016s == 20200924185945 AddMoreToSorn: migrated (0.0016s) =========================== == 20200924190652 AddSectionsToSorn: migrating ================================ -- add_column(:sorns, :routine_uses, :string) -> 0.0025s -- add_column(:sorns, :retention, :string) -> 0.0007s -- add_column(:sorns, :exemptions, :string) -> 0.0009s == 20200924190652 AddSectionsToSorn: migrated (0.0042s) ======================= == 20200925172818 AddAllToSorn: migrating ===================================== -- add_column(:sorns, :summary, :string) -> 0.0036s -- add_column(:sorns, :dates, :string) -> 0.0034s -- add_column(:sorns, :addresses, :string) -> 0.0012s -- add_column(:sorns, :further_info, :string) -> 0.0006s -- add_column(:sorns, :supplementary_info, :string) -> 0.0008s -- add_column(:sorns, :security, :string) -> 0.0010s -- add_column(:sorns, :location, :string) -> 0.0015s -- add_column(:sorns, :manager, :string) -> 0.0008s -- add_column(:sorns, :categories_of_individuals, :string) -> 0.0006s -- add_column(:sorns, :source, :string) -> 0.0004s -- add_column(:sorns, :storage, :string) -> 0.0008s -- add_column(:sorns, :retrieval, :string) -> 0.0023s -- add_column(:sorns, :safeguards, :string) -> 0.0010s -- add_column(:sorns, :access, :string) -> 0.0004s -- add_column(:sorns, :contesting, :string) -> 0.0007s -- add_column(:sorns, :notification, :string) -> 0.0010s -- remove_column(:sorns, :sections) -> 0.0022s == 20200925172818 AddAllToSorn: migrated (0.0227s) ============================ == 20200925213051 AddHeadersToSorn: migrating ================================= -- add_column(:sorns, :headers, :string) -> 0.0034s == 20200925213051 AddHeadersToSorn: migrated (0.0034s) ======================== == 20200930173010 AddSystemNumberToSorn: migrating ============================ -- add_column(:sorns, :system_number, :string) -> 0.0018s -- rename_column(:sorns, :system_name_and_number, :system_name) -> 0.0070s == 20200930173010 AddSystemNumberToSorn: migrated (0.0088s) =================== == 20201001222130 AddDataSourceToAgency: migrating ============================ -- add_column(:agencies, :data_source, :string) -> 0.0025s == 20201001222130 AddDataSourceToAgency: migrated (0.0025s) =================== == 20201001222141 AddDataSourceToSorn: migrating ============================== -- add_column(:sorns, :data_source, :string) -> 0.0015s == 20201001222141 AddDataSourceToSorn: migrated (0.0015s) ===================== == 20201002165508 AddCitationToSorn: migrating ================================ -- add_column(:sorns, :citation, :string) -> 0.0017s == 20201002165508 AddCitationToSorn: migrated (0.0018s) ======================= == 20201008224246 AddIndexesToEverything: migrating =========================== -- execute("create index action_idx on sorns using gist(to_tsvector('english', action));") -> 0.0070s -- execute("create index summary_idx on sorns using gist(to_tsvector('english', summary));") -> 0.0027s -- execute("create index dates_idx on sorns using gist(to_tsvector('english', dates));") -> 0.0015s -- execute("create index addresses_idx on sorns using gist(to_tsvector('english', addresses));") -> 0.0012s -- execute("create index further_info_idx on sorns using gist(to_tsvector('english', further_info));") -> 0.0018s -- execute("create index supplementary_info_idx on sorns using gist(to_tsvector('english', supplementary_info));") -> 0.0040s -- execute("create index system_name_idx on sorns using gist(to_tsvector('english', system_name));") -> 0.0031s -- execute("create index system_number_idx on sorns using gist(to_tsvector('english', system_number));") -> 0.0028s -- execute("create index security_idx on sorns using gist(to_tsvector('english', security));") -> 0.0014s -- execute("create index location_idx on sorns using gist(to_tsvector('english', location));") -> 0.0014s -- execute("create index manager_idx on sorns using gist(to_tsvector('english', manager));") -> 0.0038s -- execute("create index authority_idx on sorns using gist(to_tsvector('english', authority));") -> 0.0024s -- execute("create index purpose_idx on sorns using gist(to_tsvector('english', purpose));") -> 0.0020s -- execute("create index categories_of_individuals_idx on sorns using gist(to_tsvector('english', categories_of_individuals));") -> 0.0022s -- execute("create index categories_of_record_idx on sorns using gist(to_tsvector('english', categories_of_record));") -> 0.0033s -- execute("create index source_idx on sorns using gist(to_tsvector('english', source));") -> 0.0024s -- execute("create index routine_uses_idx on sorns using gist(to_tsvector('english', routine_uses));") -> 0.0020s -- execute("create index storage_idx on sorns using gist(to_tsvector('english', storage));") -> 0.0011s -- execute("create index retrieval_idx on sorns using gist(to_tsvector('english', retrieval));") -> 0.0020s -- execute("create index retention_idx on sorns using gist(to_tsvector('english', retention));") -> 0.0023s -- execute("create index safeguards_idx on sorns using gist(to_tsvector('english', safeguards));") -> 0.0023s -- execute("create index access_idx on sorns using gist(to_tsvector('english', access));") -> 0.0018s -- execute("create index contesting_idx on sorns using gist(to_tsvector('english', contesting));") -> 0.0023s -- execute("create index notification_idx on sorns using gist(to_tsvector('english', notification));") -> 0.0024s -- execute("create index exemptions_idx on sorns using gist(to_tsvector('english', exemptions));") -> 0.0024s -- execute("create index history_idx on sorns using gist(to_tsvector('english', history));") -> 0.0025s == 20201008224246 AddIndexesToEverything: migrated (0.0653s) ================== == 20201014234853 AddCitationIndex: migrating ================================= -- add_index(:sorns, :citation) -> 0.0029s == 20201014234853 AddCitationIndex: migrated (0.0029s) ======================== == 20201016205652 DropAgencyTable: migrating ================================== -- remove_column(:sorns, :agency_id, :references) -> 0.0030s -- drop_table(:agencies) -> 0.0016s -- add_column(:sorns, :agency_names, :string) -> 0.0008s -- execute("create index agency_names_idx on sorns using gist(to_tsvector('english', agency_names));") -> 0.0018s == 20201016205652 DropAgencyTable: migrated (0.0073s) ========================= == 20201016220441 AddXmlToSorn: migrating ===================================== -- add_column(:sorns, :xml, :xml) -> 0.0048s == 20201016220441 AddXmlToSorn: migrated (0.0048s) ============================ == 20201021172347 AddApiFieldsToSorn: migrating =============================== -- add_column(:sorns, :api_action, :string) -> 0.0021s -- add_column(:sorns, :api_dates, :string) -> 0.0008s -- add_column(:sorns, :pdf_url, :string) -> 0.0006s -- add_column(:sorns, :text_url, :string) -> 0.0014s -- add_column(:sorns, :publication_date, :date) -> 0.0007s -- add_column(:sorns, :title, :string) -> 0.0017s == 20201021172347 AddApiFieldsToSorn: migrated (0.0074s) ====================== == 20201022000709 ChangeDateToStringOnSorn: migrating ========================= -- change_column(:sorns, :publication_date, :string) -> 0.0426s == 20201022000709 ChangeDateToStringOnSorn: migrated (0.0426s) ================ == 20201022220018 RemoveApiActionAndApiDatesFromSorn: migrating =============== -- remove_column(:sorns, :api_action, :string) -> 0.0022s -- remove_column(:sorns, :api_dates, :string) -> 0.0008s == 20201022220018 RemoveApiActionAndApiDatesFromSorn: migrated (0.0031s) ====== == 20201022232920 CreateAgency: migrating ===================================== -- create_table(:agencies) -> 0.0184s == 20201022232920 CreateAgency: migrated (0.0184s) ============================ == 20201022234350 CreateAgenciesSorns: migrating ============================== -- create_table(:agencies_sorns) -> 0.0087s == 20201022234350 CreateAgenciesSorns: migrated (0.0088s) ===================== == 20201110215925 AddActionTypeToSorn: migrating ============================== -- add_column(:sorns, :action_type, :string) -> 0.0020s == 20201110215925 AddActionTypeToSorn: migrated (0.0020s) ===================== == 20201113200542 ChangeMentionedOnSorns: migrating =========================== -- create_table(:mentions, {:id=>false}) -> 0.0023s -- add_index(:mentions, [:sorn_id, :mentioned_sorn_id], {:unique=>true}) -> 0.0017s -- add_index(:mentions, [:mentioned_sorn_id, :sorn_id], {:unique=>true}) -> 0.0038s == 20201113200542 ChangeMentionedOnSorns: migrated (0.0080s) ================== == 20201228232801 AddShortNameToAgency: migrating ============================= -- add_column(:agencies, :short_name, :string) -> 0.0019s == 20201228232801 AddShortNameToAgency: migrated (0.0020s) ==================== == 20210107230007 AddTsvectorColumns: migrating =============================== -- execute(" ALTER TABLE sorns\n ADD COLUMN agency_names_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.agency_names, ''))) STORED,\n ADD COLUMN action_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.action, ''))) STORED,\n ADD COLUMN summary_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.summary, ''))) STORED,\n ADD COLUMN dates_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.dates, ''))) STORED,\n ADD COLUMN addresses_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.addresses, ''))) STORED,\n ADD COLUMN further_info_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.further_info, ''))) STORED,\n ADD COLUMN supplementary_info_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.supplementary_info, ''))) STORED,\n ADD COLUMN system_name_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.system_name, ''))) STORED,\n ADD COLUMN system_number_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.system_number, ''))) STORED,\n ADD COLUMN security_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.security, ''))) STORED,\n ADD COLUMN location_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.location, ''))) STORED,\n ADD COLUMN manager_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.manager, ''))) STORED,\n ADD COLUMN authority_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.authority, ''))) STORED,\n ADD COLUMN purpose_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.purpose, ''))) STORED,\n ADD COLUMN categories_of_individuals_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.categories_of_individuals, ''))) STORED,\n ADD COLUMN categories_of_record_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.categories_of_record, ''))) STORED,\n ADD COLUMN source_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.source, ''))) STORED,\n ADD COLUMN routine_uses_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.routine_uses, ''))) STORED,\n ADD COLUMN storage_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.storage, ''))) STORED,\n ADD COLUMN retrieval_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.retrieval, ''))) STORED,\n ADD COLUMN retention_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.retention, ''))) STORED,\n ADD COLUMN safeguards_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.safeguards, ''))) STORED,\n ADD COLUMN access_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.access, ''))) STORED,\n ADD COLUMN contesting_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.contesting, ''))) STORED,\n ADD COLUMN notification_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.notification, ''))) STORED,\n ADD COLUMN exemptions_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.exemptions, ''))) STORED,\n ADD COLUMN history_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.history, ''))) STORED;\n") -> 0.0589s == 20210107230007 AddTsvectorColumns: migrated (0.0592s) ====================== == 20210107231530 AddVectorIndexes: migrating ================================= -- add_index(:sorns, :agency_names_tsvector, {:using=>:gin}) -> 0.0030s -- add_index(:sorns, :action_tsvector, {:using=>:gin}) -> 0.0021s -- add_index(:sorns, :summary_tsvector, {:using=>:gin}) -> 0.0024s -- add_index(:sorns, :dates_tsvector, {:using=>:gin}) -> 0.0017s -- add_index(:sorns, :addresses_tsvector, {:using=>:gin}) -> 0.0038s -- add_index(:sorns, :further_info_tsvector, {:using=>:gin}) -> 0.0037s -- add_index(:sorns, :supplementary_info_tsvector, {:using=>:gin}) -> 0.0057s -- add_index(:sorns, :system_name_tsvector, {:using=>:gin}) -> 0.0064s -- add_index(:sorns, :system_number_tsvector, {:using=>:gin}) -> 0.0015s -- add_index(:sorns, :security_tsvector, {:using=>:gin}) -> 0.0016s -- add_index(:sorns, :location_tsvector, {:using=>:gin}) -> 0.0026s -- add_index(:sorns, :manager_tsvector, {:using=>:gin}) -> 0.0038s -- add_index(:sorns, :authority_tsvector, {:using=>:gin}) -> 0.0029s -- add_index(:sorns, :purpose_tsvector, {:using=>:gin}) -> 0.0020s -- add_index(:sorns, :categories_of_individuals_tsvector, {:using=>:gin}) -> 0.0022s -- add_index(:sorns, :categories_of_record_tsvector, {:using=>:gin}) -> 0.0034s -- add_index(:sorns, :source_tsvector, {:using=>:gin}) -> 0.0033s -- add_index(:sorns, :routine_uses_tsvector, {:using=>:gin}) -> 0.0030s -- add_index(:sorns, :storage_tsvector, {:using=>:gin}) -> 0.0015s -- add_index(:sorns, :retrieval_tsvector, {:using=>:gin}) -> 0.0014s -- add_index(:sorns, :retention_tsvector, {:using=>:gin}) -> 0.0022s -- add_index(:sorns, :safeguards_tsvector, {:using=>:gin}) -> 0.0036s -- add_index(:sorns, :access_tsvector, {:using=>:gin}) -> 0.0021s -- add_index(:sorns, :contesting_tsvector, {:using=>:gin}) -> 0.0018s -- add_index(:sorns, :notification_tsvector, {:using=>:gin}) -> 0.0022s -- add_index(:sorns, :exemptions_tsvector, {:using=>:gin}) -> 0.0021s -- add_index(:sorns, :history_tsvector, {:using=>:gin}) -> 0.0018s == 20210107231530 AddVectorIndexes: migrated (0.0749s) ======================== == 20210108002822 CreateFullSorn: migrating =================================== -- create_view(:full_sorn_searches, {:materialized=>true}) -> 0.0220s -- add_index(:full_sorn_searches, :full_sorn_tsvector, {:using=>:gin}) -> 0.0020s == 20210108002822 CreateFullSorn: migrated (0.0240s) ========================== == 20210129220248 RemoveUnusedColumns: migrating ============================== -- remove_column(:sorns, :data_source, :string) -> 0.0018s -- remove_column(:sorns, :pdf_url, :string) -> 0.0009s -- remove_column(:sorns, :headers, :string) -> 0.0012s -- remove_column(:sorns, :text_url, :string) -> 0.0007s == 20210129220248 RemoveUnusedColumns: migrated (0.0047s) ===================== == 20210204202244 RemoveGistIndicies: migrating =============================== -- index_name_exists?(:sorns, :access_idx) -> 0.0023s -- remove_index(:sorns, {:name=>:access_idx}) -> 0.0013s -- index_name_exists?(:sorns, :action_idx) -> 0.0014s -- remove_index(:sorns, {:name=>:action_idx}) -> 0.0007s -- index_name_exists?(:sorns, :addresses_idx) -> 0.0007s -- remove_index(:sorns, {:name=>:addresses_idx}) -> 0.0012s -- index_name_exists?(:sorns, :agency_names_idx) -> 0.0026s -- remove_index(:sorns, {:name=>:agency_names_idx}) -> 0.0011s -- index_name_exists?(:sorns, :authority_idx) -> 0.0014s -- remove_index(:sorns, {:name=>:authority_idx}) -> 0.0008s -- index_name_exists?(:sorns, :categories_of_individuals_idx) -> 0.0009s -- remove_index(:sorns, {:name=>:categories_of_individuals_idx}) -> 0.0010s -- index_name_exists?(:sorns, :categories_of_record_idx) -> 0.0009s -- remove_index(:sorns, {:name=>:categories_of_record_idx}) -> 0.0014s -- index_name_exists?(:sorns, :contesting_idx) -> 0.0008s -- remove_index(:sorns, {:name=>:contesting_idx}) -> 0.0008s -- index_name_exists?(:sorns, :dates_idx) -> 0.0007s -- remove_index(:sorns, {:name=>:dates_idx}) -> 0.0006s -- index_name_exists?(:sorns, :exemptions_idx) -> 0.0005s -- remove_index(:sorns, {:name=>:exemptions_idx}) -> 0.0008s -- index_name_exists?(:sorns, :further_info_idx) -> 0.0016s -- remove_index(:sorns, {:name=>:further_info_idx}) -> 0.0007s -- index_name_exists?(:sorns, :history_idx) -> 0.0011s -- remove_index(:sorns, {:name=>:history_idx}) -> 0.0010s -- index_name_exists?(:sorns, :location_idx) -> 0.0018s -- remove_index(:sorns, {:name=>:location_idx}) -> 0.0006s -- index_name_exists?(:sorns, :manager_idx) -> 0.0008s -- remove_index(:sorns, {:name=>:manager_idx}) -> 0.0009s -- index_name_exists?(:sorns, :notification_idx) -> 0.0010s -- remove_index(:sorns, {:name=>:notification_idx}) -> 0.0024s -- index_name_exists?(:sorns, :purpose_idx) -> 0.0009s -- remove_index(:sorns, {:name=>:purpose_idx}) -> 0.0010s -- index_name_exists?(:sorns, :retention_idx) -> 0.0013s -- remove_index(:sorns, {:name=>:retention_idx}) -> 0.0024s -- index_name_exists?(:sorns, :retrieval_idx) -> 0.0008s -- remove_index(:sorns, {:name=>:retrieval_idx}) -> 0.0011s -- index_name_exists?(:sorns, :routine_uses_idx) -> 0.0013s -- remove_index(:sorns, {:name=>:routine_uses_idx}) -> 0.0017s -- index_name_exists?(:sorns, :safeguards_idx) -> 0.0009s -- remove_index(:sorns, {:name=>:safeguards_idx}) -> 0.0006s -- index_name_exists?(:sorns, :security_idx) -> 0.0014s -- remove_index(:sorns, {:name=>:security_idx}) -> 0.0006s -- index_name_exists?(:sorns, :source_idx) -> 0.0006s -- remove_index(:sorns, {:name=>:source_idx}) -> 0.0009s -- index_name_exists?(:sorns, :storage_idx) -> 0.0022s -- remove_index(:sorns, {:name=>:storage_idx}) -> 0.0014s -- index_name_exists?(:sorns, :summary_idx) -> 0.0008s -- remove_index(:sorns, {:name=>:summary_idx}) -> 0.0011s -- index_name_exists?(:sorns, :supplementary_info_idx) -> 0.0008s -- remove_index(:sorns, {:name=>:supplementary_info_idx}) -> 0.0017s -- index_name_exists?(:sorns, :system_name_idx) -> 0.0011s -- remove_index(:sorns, {:name=>:system_name_idx}) -> 0.0014s -- index_name_exists?(:sorns, :system_number_idx) -> 0.0009s -- remove_index(:sorns, {:name=>:system_number_idx}) -> 0.0027s == 20210204202244 RemoveGistIndicies: migrated (0.0660s) ====================== == 20210303173553 XmlToString: migrating ====================================== -- change_column(:sorns, :xml, :string) -> 0.0019s -- execute(" ALTER TABLE sorns\n ADD COLUMN xml_tsvector tsvector GENERATED ALWAYS AS (to_tsvector('english', coalesce(sorns.xml, ''))) STORED\n") -> 0.0406s -- add_index(:sorns, :xml_tsvector, {:using=>:gin}) -> 0.0020s == 20210303173553 XmlToString: migrated (0.0447s) ============================= == 20210303182701 RemoveSearchView: migrating ================================= -- remove_index(:full_sorn_searches, :full_sorn_tsvector) -> 0.0043s -- drop_view(:full_sorn_searches, {:materialized=>true}) -> 0.0014s == 20210303182701 RemoveSearchView: migrated (0.0057s) ======================== == 20240717224231 AddActiveJobIdConcurrencyKeyCronKeyToGoodJobs: migrating ==== -- add_column(:good_jobs, :active_job_id, :uuid) -> 0.0008s -- add_column(:good_jobs, :concurrency_key, :text) -> 0.0013s -- add_column(:good_jobs, :cron_key, :text) -> 0.0010s == 20240717224231 AddActiveJobIdConcurrencyKeyCronKeyToGoodJobs: migrated (0.0068s) == 20240717224232 AddActiveJobIdIndexAndConcurrencyKeyIndexToGoodJobs: migrating -- add_index(:good_jobs, [:active_job_id, :created_at], {:algorithm=>:concurrently, :name=>:index_good_jobs_on_active_job_id_and_created_at}) -> 0.0039s -- add_index(:good_jobs, :concurrency_key, {:where=>"(finished_at IS NULL)", :algorithm=>:concurrently, :name=>:index_good_jobs_on_concurrency_key_when_unfinished}) -> 0.0049s -- add_index(:good_jobs, [:cron_key, :created_at], {:algorithm=>:concurrently, :name=>:index_good_jobs_on_cron_key_and_created_at}) -> 0.0031s rake aborted! StandardError: An error has occurred, all later migrations canceled: (StandardError) PG::UndefinedColumn: ERROR: column good_jobs.retried_good_job_id does not exist LINE 1: ...good_jobs"."active_job_id" FROM "good_jobs" WHERE "good_jobs... ^ /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `exec_params' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:671:in `block in exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:670:in `exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:649:in `execute_and_clear' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:72:in `exec_delete' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/database_statements.rb:179:in `update' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/query_cache.rb:22:in `update' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/relation.rb:466:in `update_all' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:27:in `block (3 levels) in change' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:26:in `loop' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:26:in `block (2 levels) in change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:752:in `up' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:23:in `block in change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:788:in `block in reversible' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1016:in `execute_block' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:788:in `reversible' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:22:in `change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:867:in `exec_migration' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:851:in `block (2 levels) in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:850:in `block in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:849:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1037:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1329:in `block in execute_migration_in_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1382:in `ddl_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1328:in `execute_migration_in_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1302:in `each' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1302:in `migrate_without_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1251:in `block in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1401:in `block in with_advisory_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1416:in `block in with_advisory_lock_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1416:in `with_advisory_lock_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1397:in `with_advisory_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1251:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1086:in `up' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1061:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/tasks/database_tasks.rb:237:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:92:in `block (3 levels) in
' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:90:in `each' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:90:in `block (2 levels) in
' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:10:in `block in fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:10:in `block in fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:27:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:27:in `fork' :136:in `require' :136:in `require' -e:1:in `
' Caused by: ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR: column good_jobs.retried_good_job_id does not exist (ActiveRecord::StatementInvalid) LINE 1: ...good_jobs"."active_job_id" FROM "good_jobs" WHERE "good_jobs... ^ /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `exec_params' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:671:in `block in exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:670:in `exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:649:in `execute_and_clear' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:72:in `exec_delete' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/database_statements.rb:179:in `update' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/query_cache.rb:22:in `update' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/relation.rb:466:in `update_all' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:27:in `block (3 levels) in change' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:26:in `loop' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:26:in `block (2 levels) in change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:752:in `up' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:23:in `block in change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:788:in `block in reversible' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1016:in `execute_block' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:788:in `reversible' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:22:in `change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:867:in `exec_migration' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:851:in `block (2 levels) in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:850:in `block in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:849:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1037:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1329:in `block in execute_migration_in_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1382:in `ddl_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1328:in `execute_migration_in_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1302:in `each' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1302:in `migrate_without_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1251:in `block in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1401:in `block in with_advisory_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1416:in `block in with_advisory_lock_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1416:in `with_advisory_lock_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1397:in `with_advisory_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1251:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1086:in `up' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1061:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/tasks/database_tasks.rb:237:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:92:in `block (3 levels) in
' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:90:in `each' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:90:in `block (2 levels) in
' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:10:in `block in fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:10:in `block in fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:27:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:27:in `fork' :136:in `require' :136:in `require' -e:1:in `
' Caused by: PG::UndefinedColumn: ERROR: column good_jobs.retried_good_job_id does not exist (PG::UndefinedColumn) LINE 1: ...good_jobs"."active_job_id" FROM "good_jobs" WHERE "good_jobs... ^ /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `exec_params' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:671:in `block in exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:670:in `exec_no_cache' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql_adapter.rb:649:in `execute_and_clear' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/postgresql/database_statements.rb:72:in `exec_delete' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/database_statements.rb:179:in `update' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/query_cache.rb:22:in `update' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/relation.rb:466:in `update_all' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:27:in `block (3 levels) in change' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:26:in `loop' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:26:in `block (2 levels) in change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:752:in `up' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:23:in `block in change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:788:in `block in reversible' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1016:in `execute_block' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:788:in `reversible' /Users/bretamogilefsky/Documents/Code/work/all_sorns/db/migrate/20240717224232_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb:22:in `change' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:867:in `exec_migration' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:851:in `block (2 levels) in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:850:in `block in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:849:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1037:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1329:in `block in execute_migration_in_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1382:in `ddl_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1328:in `execute_migration_in_transaction' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1302:in `each' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1302:in `migrate_without_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1251:in `block in migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1401:in `block in with_advisory_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1416:in `block in with_advisory_lock_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1416:in `with_advisory_lock_connection' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1397:in `with_advisory_lock' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1251:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1086:in `up' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/migration.rb:1061:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/tasks/database_tasks.rb:237:in `migrate' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:92:in `block (3 levels) in
' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:90:in `each' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activerecord-6.1.7.8/lib/active_record/railties/databases.rake:90:in `block (2 levels) in
' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:10:in `block in fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:10:in `block in fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:27:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:8:in `fork' /Users/bretamogilefsky/.gem/ruby/3.2.4/gems/activesupport-6.1.7.8/lib/active_support/fork_tracker.rb:27:in `fork' :136:in `require' :136:in `require' -e:1:in `
' Tasks: TOP => db:migrate:reset => db:migrate (See full trace by running task with --trace) ```
bensheldon commented 3 months ago

That's weird. I'm not sure why that migration would be looking for the column. This is the original migration and it doesn't have a reference to good_jobs.retried_good_job_id:

https://github.com/bensheldon/good_job/blame/v1.99.0/lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb#L24-L32

The good_jobs.retried_good_job_id column is added in the next migration:

https://github.com/bensheldon/good_job/tree/v1.99.0/lib/generators/good_job/templates/update/migrations

Both of these migrations were introduced in GoodJob v1.x.

mogul commented 3 months ago

Help me validate my steps...?

  1. The app started on 1.99. I applied all the v1 upgrades in preparation for migrating to v2, and merged that PR to main so the schema was ready for v2 in production. That went fine.
  2. I did the same process to upgrade from v1.99 to v2.99 to get on 2.x and prepare for upgrading to v3. I merged that branch, which rolled out to production and appears to be working fine.
  3. I tried to incrementally go from 2.99 to 3.99, resulting in the problem I'm documenting here.

Did I miss something by upgrading the good_job gem straight from 1.99 to 2.99, and then doing rails g good_job:update, in Step 2? 🤔

mogul commented 3 months ago

That migration was definitely in place as I prepared to upgrade to 2.x. 🤔

bensheldon commented 3 months ago

Oops, that link is broken.

It should only be necessary to go from v1.99 to v2.99 to v3.99 running rails g good_job:update at each one when upgrading.

Can you doublecheck the content of your migration? It should be the GoodJobJob class being operated on; maybe it got changed to GoodJob::Job? Otherwise I have no idea why that migration would be referencing that column.

mogul commented 3 months ago

(I fixed the link above.)

OK, mine definitely looks different. Yours:

    add_index :good_jobs, [:active_job_id, :created_at], algorithm: :concurrently, name: :index_good_jobs_on_active_job_id_and_created_at
    add_index :good_jobs, :concurrency_key, where: "(finished_at IS NULL)", algorithm: :concurrently, name: :index_good_jobs_on_concurrency_key_when_unfinished
    add_index :good_jobs, [:cron_key, :created_at], algorithm: :concurrently, name: :index_good_jobs_on_cron_key_and_created_at

    reversible do |dir|
      dir.up do
        start_time = Time.current
        loop do
          break if GoodJobJobs.where(active_job_id: nil, finished_at: nil).where("created_at < ?", start_time).limit(UPDATE_BATCH_SIZE).update_all("active_job_id = (serialized_params->>'job_id')::uuid").zero?
        end
      end
    end

Mine (my added comments preceded by ###):

    add_index :good_jobs, [:active_job_id, :created_at], algorithm: :concurrently, name: :index_good_jobs_on_active_job_id_and_created_at
    add_index :good_jobs, :concurrency_key, where: "(finished_at IS NULL)", algorithm: :concurrently, name: :index_good_jobs_on_concurrency_key_when_unfinished
    add_index :good_jobs, [:cron_key, :created_at], algorithm: :concurrently, name: :index_good_jobs_on_cron_key_and_created_at

    return unless defined? GoodJob::Job. ### Where did this line come from?!

    reversible do |dir|
      dir.up do
        # Ensure that all `good_jobs` records have an active_job_id value
        start_time = Time.current
        loop do
          ### Why does this refer to GoodJob::Job?!
          break if GoodJob::Job.where(active_job_id: nil, finished_at: nil).where("created_at < ?", start_time).limit(UPDATE_BATCH_SIZE).update_all("active_job_id = (serialized_params->>'job_id')::uuid").zero? 
        end
      end
    end

I'll try merging the diff into mine and see if it fixes it.

mogul commented 3 months ago

I'm unable to test this change locally because I've somehow managed to b0rk my local environment, but I pushed the commit up to the PR and it's passing tests again now!

I would love to understand how I got this slightly different migration before I close this out... Any ideas?

bensheldon commented 3 months ago

Oh! Ok, so this is weird and definitely something wrong with GoodJob v1.99. It looks like that is the file in that release: https://github.com/bensheldon/good_job/pull/353/files#diff-8bb4acb4ca9498957130c5c8bb573ecd53522b9262535d368032c370d9ec1336R27

...but now I'm totally confused as to why I made that change to the migration file. The previous one with the custom job class is more durable to exactly this kind of problem 🤔

bensheldon commented 3 months ago

I just pushed up v1.99.2 with the fixed migration: https://github.com/bensheldon/good_job/releases/tag/v1.99.2

Sorry about that!

mogul commented 3 months ago

Thanks for that, and I'm happy to have helped find a bug so that others don't trip on it!

Unfortunately I'm now running smack into that 6-1-STABLE thing, which has a simple enough workaround... except that a gem reference to a GitHub repo can't do shallow copies, and the added size of the Rails GitHub repo blows past a size limit on the platform that I'm deploying to. 😞 I'll keep working to update to Rails 7 locally.

bensheldon commented 3 months ago

Unfortunately I'm now running smack into that 6-1-STABLE thing....

Oh no! 😦 I do appreciate you reporting the problem you ran into. I hope you get the upgrade sorted easily (I don't remember 6.1 -> 7.0 being too terrible).