thiagopradi / octopus

Database Sharding for ActiveRecord
2.53k stars 505 forks source link

Migrations pending on master database #428

Closed jpbalarini closed 10 months ago

jpbalarini commented 7 years ago

I'm having a problem with pending migrations on my master database. I get the Migrations are pending; run 'bin/rake db:migrate RAILS_ENV=development' to resolve this issue. error.

Some background about what I want. I have millions of records on the Employees and Companies tables that I want to move to sharded databases. My idea is to separate records from those 2 tables to sharded tables but divided by country. In the master database, I still want to have every Employee and Company (not divided by country). So, for example, if I have and Employee from Uruguay and another one from Brazil I would like to have the Uruguayan one on the master database (specified by database.yml) and in the Uruguayan shard. For the Brazilian one, it would go to the master database and to the Brazilian shard.

This is my shards.yml:

octopus:
  environments:
    - development
  development:
    country_shards:
      uy:
        adapter: postgresql
        database: app_uy
      br:
        adapter: postgresql
        database: app_br

and this is my database.yml:

default: &default
  adapter: postgresql
  encoding: unicode
  pool: 5
  username: postgres
  password:

development:
  <<: *default
  database: app_master
  host: localhost

For achieving this, I had to create 2 migrations for the Employees table and 2 for the Companies. For example for Employees my migrations are like:

**Create employees migration (goes to master database)**
class CreateEmployees < ActiveRecord::Migration
  def change
    create_table :employees, id: :uuid do |t|
      t.string :name
      t.string :country
    end
  end
end

**Create employees sharded migration (goes to sharded databases)**
class CreateEmployees < ActiveRecord::Migration
  using_group(:country_shards)

  def change
    create_table :employees, id: :uuid do |t|
      t.string :name
      t.string :country
    end
  end
end

The problem is that migrations that run in the group country_shards are not added to the schema_migrations table for the master database. So when I run the server I get the Migrations are pending; run 'bin/rake db:migrate RAILS_ENV=development' to resolve this issue. error.

If I do a:

rake db:migrate:status

I can see that indeed are not in the database:

  up     20160518181910  Create employees
  down    20160518181911  Create employees sharded

Is this a bug? Why is this happening? Thanks in advance!

coderhs commented 7 years ago

@jpbalarini did you find a solution for this?

jpbalarini commented 7 years ago

Nope, I had to disable the migrations are pending error with:

config.active_record.migration_error = false

which isn't a good practice.

aks commented 7 years ago

FYI. To get migrations done correctly on the master, with a correct schema dump, I had to disable the replica before the migration, and re-enable it after.

Would be good if Octopus just migrated master only by default, or had a switch to enable this.

Freika commented 7 years ago

Same issue here.