Database migrations

[rachel3834]

Etienne has been trying to implement reception_data.py under the crontab on einstein and has hit a problem trying to ingest data on new images. We traced this to the fact that the deployed pipeline hasn't been migrated to incorporate changes to the model.py file (that is, apply changes to the Django definition of the DB tables to the actual SQLite database). When I tried to apply this migration, I ran into errors with Tmax.

This suggests that a previous migration hasn't been properly applied.

[rachel3834]

Usually this kind of error is resolved by deleting the previous migration files, and re-running makemigrations. As a prelude to this I updated the git repo and docker container from their hubs. When I restarted the container, the DB appears to work fine and when I tried to makemigrations the code reported no migrations to make. This suggests the issue was resolved by simply ensuring the local code repo was up to date with the active DB.

Etienne, please continue your tests with reception_data.py and let me know if you see any further issues.

[ebachelet]

I tried my code, he refused to ingest inside the db. Checking the image section of the admin, it still shows the image.shiftx column error. I guess something still wrong with migrations.

[rachel3834]

Thanks Etienne. You're right - the migrations had not deployed properly. Django kept reporting that no changes (to the DB model) had been made and hence no migration was necessary.
I used python manage.py inspectdb to view the table structure of the actual DB in active use and found that the shiftx, shifty parameters were NOT yet included in the Image table, so the error is correct but the model.py file did include them.

I verified that the git repo on einstein was on master and up to date and confirmed that the container build was properly constructed from that repo. No changes.

By making minor changes to the models.py file (removing and then re-adding the shiftx, shifty parameters with different names: shift_x, shift_y) I was able to make the DB on my laptop perform the migration, after which the DB appears normal, and the admin view can load the Image table OK.

It took a long time to get the install on einstein to accept these changes. I even migrated the DB on my laptop and rsynced this version to Einstein. Eventually the problem seemed to be that the docker container wasn't properly restarting (meaning that the code running the webservice was out of date despite claiming to have updated). I forced a docker-compose stop and then restarted it with docker-compose up -d.

The live DB now appears to be operating, and I can load the admin view of the Image table. It gives me the option - correctly - to add Image entries with the shift_x and shift_y parameters.

Etienne, please try again and let us know if it works now. But please note the modified parameter names.

[ebachelet]

OK, I have update update_db_2.add_image shiftx/shifty to shift_x,shift_y, it works like a charm! Should I push this when the db is updated??

[rachel3834]

Terrific! You can go ahead and git add/commit/push your updates. I would like to do the actual deployment of the updated code to einstein, so let's do that this afternoon when you're ready.