Fail to run migrations from scratch using Django management, but all tests run

[abeedvisram]

Been running into a few issues this evening trying to run the migrations for this project. When running locally (using Postgres running on my laptop) or via Docker, I run into the following issue

Starting coreservice_core_db_1
Recreating coreservice_core_1
Recreating coreservice_nginx_1
Attaching to coreservice_core_db_1, coreservice_core_1, coreservice_nginx_1
core_1     | + python manage.py migrate -v3 --no-input
core_db_1  | LOG:  database system was shut down at 2017-01-18 01:56:53 UTC
core_db_1  | LOG:  MultiXact member wraparound protections are now enabled
core_db_1  | LOG:  database system is ready to accept connections
core_db_1  | LOG:  autovacuum launcher started
core_1     | Operations to perform:
core_1     |   Apply all migrations: api, contenttypes
core_1     | Running pre-migrate handlers for application contenttypes
core_1     | Running pre-migrate handlers for application rest_framework
core_1     | Running pre-migrate handlers for application api
core_1     | Running pre-migrate handlers for application organisms
core_1     | Running pre-migrate handlers for application genes
core_1     | Running migrations:
core_1     |   Rendering model states... DONE (0.004s)
core_db_1  | ERROR:  relation "genes_gene" does not exist
core_db_1  | STATEMENT:  ALTER TABLE "mutations" ADD CONSTRAINT "mutations_gene_id_6289b708_fk_genes_gene_id" FOREIGN KEY ("gene_id") REFERENCES "genes_gene" ("id") DEFERRABLE INITIALLY DEFERRED
core_1     |   Applying api.0001_initial...Traceback (most recent call last):
core_1     |   File "/usr/local/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
core_1     |     return self.cursor.execute(sql, params)
core_1     | psycopg2.ProgrammingError: relation "genes_gene" does not exist
core_1     | 
core_1     | 
... <truncated>

When running the tests locally the migrations all run successfully, and all the tables are created:

$ ./manage.py test -v3
Creating test database for alias 'default' ('test_cognoma_core_service')...
Operations to perform:
  Synchronize unmigrated apps: rest_framework, organisms, staticfiles, genes, postgres
  Apply all migrations: contenttypes, api
Running pre-migrate handlers for application contenttypes
Running pre-migrate handlers for application rest_framework
Running pre-migrate handlers for application api
Running pre-migrate handlers for application organisms
Running pre-migrate handlers for application genes
Synchronizing apps without migrations:
  Creating tables...
    Creating table organisms_organism
    Creating table genes_gene
    Creating table genes_crossrefdb
    Creating table genes_crossref
    Running deferred SQL...
Running migrations:
  Rendering model states... DONE (0.011s)
  Applying api.0001_initial... OK (0.356s)
  Applying api.0002_alter_sample_fields... OK (0.048s)
  Applying api.0003_genes_mutations... OK (0.094s)
  Applying contenttypes.0001_initial... OK (0.031s)
  Applying contenttypes.0002_remove_content_type_name... OK (0.033s)
Running post-migrate handlers for application contenttypes
Adding content type 'contenttypes | contenttype'
Running post-migrate handlers for application rest_framework
Running post-migrate handlers for application api
Adding content type 'api | mutation'
Adding content type 'api | user'
Adding content type 'api | disease'
Adding content type 'api | sample'
Adding content type 'api | gene'
Adding content type 'api | classifier'
Running post-migrate handlers for application organisms
Adding content type 'organisms | organism'
Running post-migrate handlers for application genes
Adding content type 'genes | gene'
Adding content type 'genes | crossref'
Adding content type 'genes | crossrefdb'
test_cannot_update_other_user_classifier (api.test.test_classifiers.ClassifierTests) ... ok
... <truncated>

To get around this, and so that I could start playing around with the project this evening, I copied the schema from the test database and set that as my local DB. Obviously not a proper way of doing things, but meant that I could hack on the project a bit.

Has anybody else encountered this? I'm sure I'm missing something really obvious with regards to the different settings between running the unit tests and the management commands, but I can't seem to figure out what the issue is. Any help much appreciated.

[awm33]

Removing the mutations and genes django apps may fix this. We do not use those anymore.

[dhimmel]

@awm33, we should use the gene table from cognoma/genes, rather than django-genes. I thought we already removed django-genes?

Also the backend database will need mutation info.

[awm33]

@dhimmel We did, the application is using the new genes table, but django is still loading django-genes and django-organisms. We can just remove it from installed applications in settings.py and requirements.txt

[dhimmel]

django is still loading django-genes and django-organisms. We can just remove it from installed applications in settings.py and requirements.txt

Purge it!