9.3.0 db:migrate:with_data doesn't work as expected

[schinery]

Having bumped from 9.2.0 to 9.3.0, the rake task db:migrate:with_data doesn't work as expected as it no longer runs schema migrations.

After running the task with pending schema and data migrations, only the data migrations have been run.

rails db:migrate:status:with_data

This outputs as:

up      schema  20240117083816 Some migration # previously run schema change
down    schema  20240125090952 Some migration
down    schema  20240130152141 Some migration
down    schema  20240130152200 Some migration
down    schema  20240130152216 Some migration
down    schema  20240203071550 Some migration
down    schema  20240205112535 Some migration
down    schema  20240208145441 Some migration
down    schema  20240208150333 Some migration
down    schema  20240213141420 Some migration
down    schema  20240214132058 Some migration
down    schema  20240215085530 Some migration
down    schema  20240215134650 Some migration
up      data    20240219103425 Some migration
down    schema  20240219113743 Some migration
down    schema  20240221090212 Some migration

After pinning the gem to 9.2.0 and re-running db:migrate:with_data we have:

up      schema  20240117083816 Some migration
up      schema  20240125090952 Some migration
up      schema  20240130152141 Some migration
up      schema  20240130152200 Some migration
up      schema  20240130152216 Some migration
up      schema  20240203071550 Some migration
up      schema  20240205112535 Some migration
up      schema  20240208145441 Some migration
up      schema  20240208150333 Some migration
up      schema  20240213141420 Some migration
up      schema  20240214132058 Some migration
up      schema  20240215085530 Some migration
up      schema  20240215134650 Some migration
up      data    20240219103425 Some migration
up      schema  20240219113743 Some migration
up      schema  20240221090212 Some migration

[ngan]

Hey @schinery -- what version of Rails are you on?

[schinery]

@ngan this is in a project using Ruby 3.3.0 and Rails 7.1.3.2.

(Sorry, should of said originally)

[ngan]

Hey @schinery we are unable to reproduce this issue on the same version of Ruby/Rails as you.

$ bin/rails db:migrate:status:with_data
Running via Spring preloader in process 41977

database: <redacted>_development

 Status    Type    Migration ID   Migration Name
------------------------------------------------------------
   up     schema  20240308192354  <redacted>
   up      data   20240308195333  <redacted>
   up     schema  20240311211645  <redacted>
  down    schema  20240311212739  <redacted>
   up      data   20240311225815  <redacted>

$ bin/rails db:migrate:with_data
Running via Spring preloader in process 42656
== 20240311212739 <redacted>: migrating ===========================
-- change(:<redacted>)
   -> 0.0249s
== 20240311212739 <redacted>: migrated (0.0252s) ==================

$ bin/rails db:migrate:status:with_data
Running via Spring preloader in process 41977

database: <redacted>_development

 Status    Type    Migration ID   Migration Name
------------------------------------------------------------
   up     schema  20240308192354  <redacted>
   up      data   20240308195333  <redacted>
   up     schema  20240311211645  <redacted>
   up     schema  20240311212739  <redacted>
   up      data   20240311225815  <redacted>

As you can see the migration 20240311212739 was migrated just fine.

Do you have any special task overrides happening for db:migrate:with_data?

[schinery]

Hey @ngan

Sorry, another piece of info that might have initially helped but I only just thought of this after you everything is working for you...

My DB schema migrations are actually in a mounted Rails engine inside the Rails app (is shared with a couple of apps using the same DB), and it appears that it is only the schema migrations from the engine that aren't being run.

To test this I've added 2 migrations, 1 named "Add foo to clients" to the Rails app and another named "Add bar to clients" to the Rails engine. After I run rails db:migrate:with_data in the Rails app and then check the status I get:

   up     schema  20240313125055  Add foo to clients
  down    schema  20240313125126  Add bar to clients

I'm trying to do some more investigation my side to see if it could be something else to do this project's setup, but as mentioned 9.2.0 works just fine.

[ngan]

Do you get the right results if you did db:migrate:status (without the data)?

[schinery]

If I run rails db:migrate:status I get the same status as :with_data.

   up     20240313125055  Add foo to clients
  down    20240313125126  Add bar to clients

However, if I run rails db:migrate without the :with_data then...

rails db:migrate

== 20240313125126 AddBarToUsers: migrating ====================================
== 20240313125126 AddBarToUsers: migrated (0.0538s) ===========================

rails db:migrate:status

   up     20240313125055  Add foo to users
   up     20240313125126  Add bar to users

I've tried a couple of the commits between 9.2.0 and 9.3.0 and think it is c8588dd4b76015ca0fa5869b21eee48bf6bb6033 that might of introduced this, but I wasn't able to use that ref as I was getting a NoMethodError: undefined method 'primary?' for nil but c5a86b4c4c16d4b1005d2ac6a81e1a0b15e29040 fixed that so I was able to run that.

[schinery]

Closing this as it appears that either 9.4.0 or something else has resolved the issue as I can't reproduce with the same tests I was using with 9.3.0.