Load initial data with data migrations

[handrews]

Calling loaddata() in migrations does not respect the migration state, so if later migrations change the schema (as one would expect them to do) the loaddata() call will try to load the old data into the old database structure, but it will attempt to use the latest schema from the python side. That, obviously, does not work.

We need to either directly inline our data into the data migrations, or have each read it from the fixtures in question, or write an Operation class that reads from fixtures and then loads through the appropriate migration instance. I'm not sure which is best yet but will research further an update here or start posting PRs linking back to this issue.

[jochengcd]

At least for some cases it seems possible to do this via editing the dependencies. That way one keeps the filenames, so no other dependencies need to be edited and the existing migration-state doesn't get confused.

At least for the currently broken stddata-app this works. There might be situations where cross-dependencies prevent this, I gather we will see once we get there.

[jochengcd]

Further thought, in some cases we should be able to edit existing initial data loading to just do nothing, and then add a later data loading for the changed model. And we don't do that much data loading anyway.

[handrews]

I think the best thing to do here is to keep the fixtures up to date with the current schema. I just submitted a PR for the only discrepancy I saw right now. If that makes sense then this can be closed out.

[jochengcd]

Keeping the fixtures up to date is right, but that doesn't solve the problem with the migrations in all cases. I.e., if we load in 000x the fixture with the current schema, but in 000x+1 we migrate the underlying model this should clash ?

I think it is a combo of keeping the fixtures up to date and rearranging the migration order by moving the initial_data script to after the migrations which affect the underlying model.

[handrews]

Oh I forgot about initial_data as a script. I just went through the Django 1.11 docs to set things up, which involved (assuming the gcdonline user was set up):

bash$ mysql -ugcdonline gcdonline <<EOD
CREATE DATABASE gcdonline DEFAULT CHARACTER SET utf8;
GRANT ALL ON gcdonline.* to 'gcdonline'@'localhost';
EOD

bash$ python manage.py migrate
bash$ python manage.py loaddata <each fixture name>

How does initial_data fit in again?