The-Compiler commented 4 years ago

Ich werde hier mal dokumentieren, wie ich bei der Migration der Daten vorgehe. Bei einem ersten (quick & dirty)-Test gab's leider schon mal einige Probleme. Ich versuche das jetzt noch ein zweites Mal etwas geordneter.

Bisher gehe ich so vor:

Daten umziehen

Media-Dateien auf neuen Server rüberkopieren und Berechtigungen anpassen
SQL-Dump: su postgres pg_dump --clean --encoding=UTF-8 studentenportal > studentenportal.pgsql (und danach rüberkopieren)

Datenbank wiederherstellen

Testdaten loswerden: dropdb -U studentenportal studentenportal (vermutlich dank pg_dump --clean eigentlich nicht gebraucht)
Leere Datenbank anlegen: createdb -T template0 -U studentenportal studentenportal (dito)
Dump laden. Mehr dazu im nächsten Post.

The-Compiler commented 4 years ago

Dump laden

Mittels psql -U studentenportal studentenportal < studentenportal.pgsql

Dabei gibt's einige Fehler

``` SET SET SET SET SET SET SET ERROR: relation "public.front_user_user_permissions" does not exist ERROR: relation "public.registration_registrationprofile" does not exist ERROR: relation "public.documents_documentrating" does not exist ERROR: relation "public.front_user_groups" does not exist ERROR: relation "public.lecturers_quotevote" does not exist ERROR: relation "public.lecturers_lecturerrating" does not exist ERROR: relation "public.django_admin_log" does not exist ERROR: relation "public.documents_document" does not exist ERROR: relation "public.easy_thumbnails_thumbnail" does not exist ERROR: relation "public.lecturers_quotevote" does not exist ERROR: relation "public.front_user_user_permissions" does not exist ERROR: relation "public.oauth2_refreshtoken" does not exist ERROR: relation "public.oauth2_refreshtoken" does not exist ERROR: relation "public.oauth2_refreshtoken" does not exist ERROR: relation "public.oauth2_grant" does not exist ERROR: relation "public.oauth2_grant" does not exist ERROR: relation "public.oauth2_client" does not exist ERROR: relation "public.oauth2_accesstoken" does not exist ERROR: relation "public.oauth2_accesstoken" does not exist ERROR: relation "public.documents_documentcategory_lecturers" does not exist ERROR: relation "public.front_user_groups" does not exist ERROR: relation "public.auth_group_permissions" does not exist ERROR: relation "public.documents_documentrating" does not exist ERROR: relation "public.documents_documentcategory_courses" does not exist ERROR: relation "public.documents_documentcategory_lecturers" does not exist ERROR: relation "public.documents_documentdownload" does not exist ERROR: relation "public.django_admin_log" does not exist ERROR: relation "public.documents_documentcategory_courses" does not exist ERROR: relation "public.auth_permission" does not exist ERROR: relation "public.documents_document" does not exist ERROR: relation "public.events_event" does not exist ERROR: relation "public.lecturers_quote" does not exist ERROR: relation "public.auth_user_user_permissions" does not exist ERROR: relation "public.auth_user_groups" does not exist ERROR: relation "public.auth_group_permissions" does not exist ERROR: index "oauth2_refreshtoken_user_id" does not exist ERROR: index "oauth2_refreshtoken_client_id" does not exist ERROR: index "oauth2_grant_user_id" does not exist ERROR: index "oauth2_grant_client_id" does not exist ERROR: index "oauth2_client_user_id" does not exist ERROR: index "oauth2_accesstoken_user_id" does not exist ERROR: index "oauth2_accesstoken_client_id" does not exist ERROR: index "lecturers_quotevote_user_id" does not exist ERROR: index "lecturers_quotevote_quote_id" does not exist ERROR: index "lecturers_quote_lecturer_id" does not exist ERROR: index "lecturers_quote_author_id" does not exist ERROR: index "lecturers_lecturerrating_user_id" does not exist ERROR: index "lecturers_lecturerrating_rating" does not exist ERROR: index "lecturers_lecturerrating_lecturer_id" does not exist ERROR: index "lecturers_lecturerrating_category_like" does not exist ERROR: index "lecturers_lecturerrating_category" does not exist ERROR: index "lecturers_course_abbreviation_like" does not exist ERROR: index "front_user_username_like" does not exist ERROR: index "front_user_user_permissions_user_id" does not exist ERROR: index "front_user_user_permissions_permission_id" does not exist ERROR: index "front_user_groups_user_id" does not exist ERROR: index "front_user_groups_group_id" does not exist ERROR: index "events_event_author_id" does not exist ERROR: index "easy_thumbnails_thumbnail_storage_hash" does not exist ERROR: index "easy_thumbnails_thumbnail_source_id" does not exist ERROR: index "easy_thumbnails_thumbnail_name" does not exist ERROR: index "easy_thumbnails_source_storage_hash" does not exist ERROR: index "easy_thumbnails_source_name" does not exist ERROR: index "documents_documentrating_user_id" does not exist ERROR: index "documents_documentrating_document_id" does not exist ERROR: index "documents_documentdownload_ip" does not exist ERROR: index "documents_documentdownload_document_id" does not exist ERROR: index "documents_documentcategory_name" does not exist ERROR: index "documents_documentcategory_lecturers_lecturer_id" does not exist ERROR: index "documents_documentcategory_lecturers_documentcategory_id" does not exist ERROR: index "documents_documentcategory_courses_documentcategory_id" does not exist ERROR: index "documents_documentcategory_courses_course_id" does not exist ERROR: index "documents_document_uploader_id" does not exist ERROR: index "documents_document_category_id" does not exist ERROR: index "django_session_expire_date" does not exist ERROR: index "django_admin_log_user_id" does not exist ERROR: index "django_admin_log_content_type_id" does not exist ERROR: index "auth_user_user_permissions_user_id" does not exist ERROR: index "auth_user_user_permissions_permission_id" does not exist ERROR: index "auth_user_groups_user_id" does not exist ERROR: index "auth_user_groups_group_id" does not exist ERROR: index "auth_permission_content_type_id" does not exist ERROR: index "auth_group_permissions_permission_id" does not exist ERROR: index "auth_group_permissions_group_id" does not exist ERROR: relation "public.south_migrationhistory" does not exist ERROR: relation "public.registration_registrationprofile" does not exist ERROR: relation "public.registration_registrationprofile" does not exist ERROR: relation "public.oauth2_refreshtoken" does not exist ERROR: relation "public.oauth2_refreshtoken" does not exist ERROR: relation "public.oauth2_grant" does not exist ERROR: relation "public.oauth2_client" does not exist ERROR: relation "public.oauth2_accesstoken" does not exist ERROR: relation "public.lecturers_quotevote" does not exist ERROR: relation "public.lecturers_quotevote" does not exist ERROR: relation "public.lecturers_quote" does not exist ERROR: relation "public.lecturers_lecturerrating" does not exist ERROR: relation "public.lecturers_lecturerrating" does not exist ERROR: relation "public.lecturers_lecturer" does not exist ERROR: relation "public.lecturers_lecturer" does not exist ERROR: relation "public.lecturers_course" does not exist ERROR: relation "public.lecturers_course" does not exist ERROR: relation "public.front_user" does not exist ERROR: relation "public.front_user_user_permissions" does not exist ERROR: relation "public.front_user_user_permissions" does not exist ERROR: relation "public.front_user" does not exist ERROR: relation "public.front_user_groups" does not exist ERROR: relation "public.front_user_groups" does not exist ERROR: relation "public.events_event" does not exist ERROR: relation "public.easy_thumbnails_thumbnail" does not exist ERROR: relation "public.easy_thumbnails_thumbnail" does not exist ERROR: relation "public.easy_thumbnails_source" does not exist ERROR: relation "public.easy_thumbnails_source" does not exist ERROR: relation "public.documents_documentrating" does not exist ERROR: relation "public.documents_documentrating" does not exist ERROR: relation "public.documents_documentdownload" does not exist ERROR: relation "public.documents_documentcategory" does not exist ERROR: relation "public.documents_documentcategory" does not exist ERROR: relation "public.documents_documentcategory_lecturers" does not exist ERROR: relation "public.documents_documentcategory_courses" does not exist ERROR: relation "public.documents_documentcategory_lecturers" does not exist ERROR: relation "public.documents_documentcategory_courses" does not exist ERROR: relation "public.documents_document" does not exist ERROR: relation "public.django_session" does not exist ERROR: relation "public.django_content_type" does not exist ERROR: relation "public.django_content_type" does not exist ERROR: relation "public.django_admin_log" does not exist ERROR: relation "public.auth_user_user_permissions" does not exist ERROR: relation "public.auth_user_user_permissions" does not exist ERROR: relation "public.auth_user_groups" does not exist ERROR: relation "public.auth_user_groups" does not exist ERROR: relation "public.auth_permission" does not exist ERROR: relation "public.auth_permission" does not exist ERROR: relation "public.auth_group" does not exist ERROR: relation "public.auth_group_permissions" does not exist ERROR: relation "public.auth_group_permissions" does not exist ERROR: relation "public.auth_group" does not exist ERROR: relation "public.south_migrationhistory" does not exist ERROR: relation "public.registration_registrationprofile" does not exist ERROR: relation "public.oauth2_refreshtoken" does not exist ERROR: relation "public.oauth2_grant" does not exist ERROR: relation "public.oauth2_client" does not exist ERROR: relation "public.oauth2_accesstoken" does not exist ERROR: relation "public.lecturers_quotevote" does not exist ERROR: relation "public.lecturers_quote" does not exist ERROR: relation "public.lecturers_lecturerrating" does not exist ERROR: relation "public.front_user_user_permissions" does not exist ERROR: relation "public.front_user_groups" does not exist ERROR: relation "public.front_user" does not exist ERROR: relation "public.events_event" does not exist ERROR: relation "public.easy_thumbnails_thumbnail" does not exist ERROR: relation "public.easy_thumbnails_source" does not exist ERROR: relation "public.documents_documentrating" does not exist ERROR: relation "public.documents_documentdownload" does not exist ERROR: relation "public.documents_documentcategory_lecturers" does not exist ERROR: relation "public.documents_documentcategory_courses" does not exist ERROR: relation "public.documents_documentcategory" does not exist ERROR: relation "public.documents_document" does not exist ERROR: relation "public.django_content_type" does not exist ERROR: relation "public.django_admin_log" does not exist ERROR: relation "public.auth_user_user_permissions" does not exist ERROR: relation "public.auth_user_groups" does not exist ERROR: relation "public.auth_permission" does not exist ERROR: relation "public.auth_group_permissions" does not exist ERROR: relation "public.auth_group" does not exist ERROR: sequence "south_migrationhistory_id_seq" does not exist ERROR: table "south_migrationhistory" does not exist ERROR: sequence "registration_registrationprofile_id_seq" does not exist ERROR: table "registration_registrationprofile" does not exist ERROR: sequence "oauth2_refreshtoken_id_seq" does not exist ERROR: table "oauth2_refreshtoken" does not exist ERROR: sequence "oauth2_grant_id_seq" does not exist ERROR: table "oauth2_grant" does not exist ERROR: sequence "oauth2_client_id_seq" does not exist ERROR: table "oauth2_client" does not exist ERROR: sequence "oauth2_accesstoken_id_seq" does not exist ERROR: table "oauth2_accesstoken" does not exist ERROR: sequence "lecturers_quotevote_id_seq" does not exist ERROR: table "lecturers_quotevote" does not exist ERROR: sequence "lecturers_quote_id_seq" does not exist ERROR: table "lecturers_quote" does not exist ERROR: sequence "lecturers_lecturerrating_id_seq" does not exist ERROR: table "lecturers_lecturerrating" does not exist ERROR: table "lecturers_lecturer" does not exist ERROR: table "lecturers_course" does not exist ERROR: sequence "front_user_user_permissions_id_seq" does not exist ERROR: table "front_user_user_permissions" does not exist ERROR: sequence "front_user_id_seq" does not exist ERROR: sequence "front_user_groups_id_seq" does not exist ERROR: table "front_user_groups" does not exist ERROR: table "front_user" does not exist ERROR: sequence "events_event_id_seq" does not exist ERROR: table "events_event" does not exist ERROR: sequence "easy_thumbnails_thumbnail_id_seq" does not exist ERROR: table "easy_thumbnails_thumbnail" does not exist ERROR: sequence "easy_thumbnails_source_id_seq" does not exist ERROR: table "easy_thumbnails_source" does not exist ERROR: sequence "documents_documentrating_id_seq" does not exist ERROR: table "documents_documentrating" does not exist ERROR: sequence "documents_documentdownload_id_seq" does not exist ERROR: table "documents_documentdownload" does not exist ERROR: sequence "documents_documentcategory_lecturers_id_seq" does not exist ERROR: table "documents_documentcategory_lecturers" does not exist ERROR: sequence "documents_documentcategory_id_seq" does not exist ERROR: sequence "documents_documentcategory_courses_id_seq" does not exist ERROR: table "documents_documentcategory_courses" does not exist ERROR: table "documents_documentcategory" does not exist ERROR: sequence "documents_document_id_seq" does not exist ERROR: table "documents_document" does not exist ERROR: table "django_session" does not exist ERROR: sequence "django_content_type_id_seq" does not exist ERROR: table "django_content_type" does not exist ERROR: sequence "django_admin_log_id_seq" does not exist ERROR: table "django_admin_log" does not exist ERROR: sequence "auth_user_user_permissions_id_seq" does not exist ERROR: table "auth_user_user_permissions" does not exist ERROR: sequence "auth_user_groups_id_seq" does not exist ERROR: table "auth_user_groups" does not exist ERROR: sequence "auth_permission_id_seq" does not exist ERROR: table "auth_permission" does not exist ERROR: sequence "auth_group_permissions_id_seq" does not exist ERROR: table "auth_group_permissions" does not exist ERROR: sequence "auth_group_id_seq" does not exist ERROR: table "auth_group" does not exist ERROR: extension "citext" does not exist DROP EXTENSION DROP SCHEMA CREATE SCHEMA ERROR: role "postgres" does not exist CREATE EXTENSION COMMENT CREATE EXTENSION COMMENT SET SET SET CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE TABLE ALTER TABLE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE CREATE TABLE ALTER TABLE CREATE SEQUENCE ALTER TABLE ALTER SEQUENCE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE COPY 0 setval -------- 1 (1 row) COPY 0 setval -------- 1 (1 row) COPY 96 setval -------- 105 (1 row) COPY 0 setval -------- 1 (1 row) COPY 0 setval -------- 1 (1 row) COPY 481 setval -------- 503 (1 row) COPY 35 setval -------- 38 (1 row) COPY 53194 COPY 1963 setval -------- 2191 (1 row) COPY 802 COPY 753 setval -------- 776 (1 row) setval -------- 1546 (1 row) COPY 732 setval -------- 775 (1 row) COPY 1532515 setval --------- 1535444 (1 row) COPY 1003 setval -------- 1025 (1 row) COPY 368 setval -------- 368 (1 row) COPY 680 setval -------- 680 (1 row) COPY 109 setval -------- 111 (1 row) COPY 4160 COPY 0 setval -------- 1 (1 row) setval -------- 4202 (1 row) COPY 0 setval -------- 1 (1 row) COPY 8 COPY 854 COPY 9028 setval -------- 9032 (1 row) COPY 1885 setval -------- 2422 (1 row) COPY 9522 setval -------- 10583 (1 row) COPY 0 setval -------- 1 (1 row) COPY 0 setval -------- 1 (1 row) COPY 0 setval -------- 1 (1 row) COPY 0 setval -------- 1 (1 row) COPY 4137 setval -------- 4165 (1 row) COPY 27 setval -------- 27 (1 row) ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE ALTER TABLE REVOKE ERROR: role "postgres" does not exist ERROR: role "postgres" does not exist GRANT ```

Ich glaube aber, dass diese ignoriert werden können. Die meisten davon sind wegen Statements, die Daten löschen, die aber gar nicht da sind. Die letzten paar sind wegen Permissions für einen postgres-User, den es im Docker-Container aber nicht mehr gibt.

Ich hab an dem Punkt einige Stichproben gemacht und die Daten sehen okay aus.

Das initiale Schema (psql -U studentenportal <<< '\d *' > schema-initial) sieht so aus:

The-Compiler commented 4 years ago

Django-Migration

Postgres-Container so zum laufen kriegen, dass der studentenportal-Container drauf zugreifen kann, aber nicht läuft
docker-compose -f docker-compose-production.yml run studentenportal /bin/bash
env-Variablen aus start.sh übernehmen
Erster Anlauf: python3 manage.py migrate front

Operations to perform:
  Apply all migrations: front
Running migrations:
  Applying contenttypes.0001_initial...Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.DuplicateTable: relation "django_content_type" already exists

Theoretisch hilft da --fake-initial (docs):

Allows Django to skip an app’s initial migration if all database tables with the names of all models created by all CreateModel operations in that migration already exist. This option is intended for use when first running migrations against a database that preexisted the use of migrations. This option does not, however, check for matching database schema beyond matching table names and so is only safe to use if you are confident that your existing schema matches what is recorded in your initial migration.

Das scheint mir aber bei uns nicht der Fall zu sein. Unser Schema passt ja nicht auf die initiale Migration, sondern da sind eigentlich schon mal Migrationen durchgelaufen (einfach via South).

In #193 sagt @dbrgn:

The first time this code is deployed on the server, the following code needs to be run: python manage.py migrate --fake

Das würde dann ja aber auch diverse Migrationen überspringen, die gar noch nicht gemacht wurden auf der aktuellen Datenbank, oder? Vermutlich müsste man da also explizit die letzte Migration angeben, die via South ausgeführt wurde? Ich grabe mal weiter!

South migration history

```sql -- -- Data for Name: south_migrationhistory; Type: TABLE DATA; Schema: public; Owner: studentenportal -- COPY south_migrationhistory (id, app_name, migration, applied) FROM stdin; 1 front 0001_initial 2014-02-25 15:07:25.681574+01 2 django_extensions 0001_empty 2014-02-25 15:07:26.298235+01 3 easy_thumbnails 0001_initial 2014-02-25 15:07:26.585375+01 4 easy_thumbnails 0002_filename_indexes 2014-02-25 15:07:26.64599+01 5 easy_thumbnails 0003_auto__add_storagenew 2014-02-25 15:07:26.735075+01 6 easy_thumbnails 0004_auto__add_field_source_storage_new__add_field_thumbnail_storage_new 2014-02-25 15:07:26.779812+01 7 easy_thumbnails 0005_storage_fks_null 2014-02-25 15:07:26.830334+01 8 easy_thumbnails 0006_copy_storage 2014-02-25 15:07:26.841954+01 9 easy_thumbnails 0007_storagenew_fks_not_null 2014-02-25 15:07:26.884035+01 10 easy_thumbnails 0008_auto__del_field_source_storage__del_field_thumbnail_storage 2014-02-25 15:07:26.89638+01 11 easy_thumbnails 0009_auto__del_storage 2014-02-25 15:07:26.907565+01 12 easy_thumbnails 0010_rename_storage 2014-02-25 15:07:26.920461+01 13 easy_thumbnails 0011_auto__add_field_source_storage_hash__add_field_thumbnail_storage_hash 2014-02-25 15:07:27.083652+01 14 easy_thumbnails 0012_build_storage_hashes 2014-02-25 15:07:27.095868+01 15 easy_thumbnails 0013_auto__del_storage__del_field_source_storage__del_field_thumbnail_stora 2014-02-25 15:07:27.13941+01 16 easy_thumbnails 0014_auto__add_unique_source_name_storage_hash__add_unique_thumbnail_name_s 2014-02-25 15:07:27.184419+01 17 easy_thumbnails 0015_auto__del_unique_thumbnail_name_storage_hash__add_unique_thumbnail_sou 2014-02-25 15:07:27.216761+01 18 oauth2 0001_initial 2014-02-25 15:07:27.629193+01 19 oauth2 0002_auto__chg_field_client_user 2014-02-25 15:07:27.671603+01 20 oauth2 0003_auto__add_field_client_name 2014-02-25 15:07:27.743344+01 21 documents 0001_initial 2014-02-25 15:07:28.247327+01 22 events 0001_initial 2014-02-25 15:07:28.401152+01 23 lecturers 0001_initial 2014-02-25 15:07:28.927696+01 24 lecturers 0002_auto__add_course 2014-06-16 23:17:56.804473+02 25 lecturers 0003_remove_modulereview 2014-06-16 23:17:56.851858+02 26 documents 0002_auto__m2m_table_for_course 2014-06-16 23:18:00.133419+02 27 documents 0003_auto__add_field_document_flattr_disabled 2014-08-12 11:57:17.718732+02 \. ```

dbrgn commented 4 years ago

Ich glaube beim Wechsel von South auf Django Migrations muss man einmal mit --fake migrieren, aber ich habe nicht mehr ganz im Kopf wie das genau läuft... Ist schon 4 Jahre her, seit ich den PR #193 erstellt hab :smile:

Im Zweifelsfall einfach mal in einer Test-DB ausprobieren.

The-Compiler commented 4 years ago

Migrations-Attempt 1

Ich versuche, die Migrationen in möglichst kleinen Schritten auf den Stand zu bringen, der momentan in der South-Tabelle steht. Dazu schaue ich mir jeweils mit --plan an, was Django an Migrationen macht, und nutze jeweils --fake mit einzelnen Apps, um auf den richtigen Stand zu bringen. Danach sollten die neuen Migrationen kein Problem mehr darstellen.

Auth

$ python3 manage.py migrate --plan auth
Planned operations:
auth.0007_alter_validators_add_error_messages
    Alter field username on user
auth.0008_alter_user_username_max_length
    Alter field username on user
auth.0009_alter_user_last_name_max_length
    Alter field last_name on user
auth.0010_alter_group_name_max_length
    Alter field name on group
auth.0011_update_proxy_permissions
    Raw Python operation ->     Update the content_type of prox…

$ python3 manage.py migrate auth
Operations to perform:
  Apply all migrations: auth
Running migrations:
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK

front

$ python3 manage.py migrate --plan --fake front 0001_initial
Planned operations:
  No planned migration operations.

$ python3 manage.py migrate --fake front 0001_initial
Operations to perform:
  Target specific migration: 0001_initial, from front
Running migrations:
  No migrations to apply.

django_extensions

Haben wir entfernt, sollte also okay sein.

easy_thumbnails

Hier mappt die south-Migration 0015_auto__del_unique_thumbnail_name_storage_hash__add_unique_thumbnail_sou nicht so wirklich auf die Django-Migrationen. Also mal gucken, was eine volle Migration ohne --fake machen würde:

$ python3 manage.py migrate --plan easy_thumbnails
Planned operations:
easy_thumbnails.0001_initial
    Create model Source
    Create model Thumbnail
    Alter unique_together for thumbnail (1 constraint(s))
    Alter unique_together for source (1 constraint(s))
easy_thumbnails.0002_thumbnaildimensions
    Create model ThumbnailDimensions

Mal probieren:

$ python3 manage.py migrate easy_thumbnails
Operations to perform:
  Apply all migrations: easy_thumbnails
Running migrations:
  Applying easy_thumbnails.0001_initial...Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.DuplicateTable: relation "easy_thumbnails_source" already exists

Okay, das initiale DB-Schema ist sicher schon hier, also mal mit --fake-initial:

$ python3 manage.py migrate easy_thumbnails --fake-initial
Operations to perform:
  Apply all migrations: easy_thumbnails
Running migrations:
  Applying easy_thumbnails.0001_initial... FAKED
  Applying easy_thumbnails.0002_thumbnaildimensions... OK
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: null value in column "name" violates not-null constraint
DETAIL:  Failing row contains (39, null, easy_thumbnails, thumbnaildimensions).

Meh. Da wurden wohl die Migrationen von easy-thumbnails mal zusammengefasst, als die von South zu Django migriert (hah) wurden...

Wenn ich mir https://github.com/SmileyChris/easy-thumbnails/commit/7360356e8d14cd3f0feac0c56ac893f960bf5c6d#diff-9f346111d1dccf8bd4fae42646e8d1ac so anschaue, dann fehlt uns nur 0016_auto__add_thumbnaildimensions.py. Die 0002_thumbnaildimensions.py welche oben failt scheint da eigentlich das Equivalent dazu zu sein...

Passiert während: STATEMENT: INSERT INTO "django_content_type" ("app_label", "model") VALUES ('easy_thumbnails', 'thumbnaildimensions') RETURNING "django_content_type"."id"

Etwas ähnlich: https://code.djangoproject.com/ticket/24299

Schlussendlich gefixt, indem ich die entsprechende contenttypes-Migration nochmals explizit durchgeführt habe:

$ python3 manage.py migrate contenttypes 0001 --fake
Operations to perform:
  Target specific migration: 0001_initial, from contenttypes
Running migrations:
  Rendering model states... DONE
  Unapplying auth.0011_update_proxy_permissions... FAKED
  Unapplying front.0001_initial... FAKED
  Unapplying auth.0010_alter_group_name_max_length... FAKED
  Unapplying auth.0009_alter_user_last_name_max_length... FAKED
  Unapplying auth.0008_alter_user_username_max_length... FAKED
  Unapplying auth.0007_alter_validators_add_error_messages... FAKED
  Unapplying auth.0006_require_contenttypes_0002... FAKED
  Unapplying contenttypes.0002_remove_content_type_name... FAKED

$ python3 manage.py migrate contenttypes
Operations to perform:
  Apply all migrations: contenttypes
Running migrations:
  Applying contenttypes.0002_remove_content_type_name... OK

$ python3 manage.py migrate easy_thumbnails 
Operations to perform:
  Apply all migrations: easy_thumbnails
Running migrations:
  No migrations to apply.

Da würde ich aber gerne noch ne schönere Lösung finden... Ev. hat einer der vorherigen Schritte implizit contenttypes.0002_remove_content_type_name als faked markiert, oder ich hatte versehentlich doch noch Resten von einem früheren Test in der Datenbank? Aber gut, mal weiter im Text...

oauth2

$ python3 manage.py migrate --plan oauth2
CommandError: No installed app with label 'oauth2'.

Vermutlich seit b2a83b2e70277d9ce94cc3738caa9ea7de91f0f5 nicht mehr relevant.

Auth, zum Zweiten

nach dem unapply:

$ python3 manage.py migrate --plan auth
Planned operations:
auth.0006_require_contenttypes_0002
auth.0007_alter_validators_add_error_messages
    Alter field username on user
auth.0008_alter_user_username_max_length
    Alter field username on user
auth.0009_alter_user_last_name_max_length
    Alter field last_name on user
auth.0010_alter_group_name_max_length
    Alter field name on group
auth.0011_update_proxy_permissions
    Raw Python operation ->     Update the content_type of prox…

$ python3 manage.py migrate auth
Operations to perform:
  Apply all migrations: auth
Running migrations:
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK

front, zum Zweiten

dito

$ python3 manage.py migrate --plan --fake front 0001_initial
Planned operations:
front.0001_initial
    Create model User

$ python3 manage.py migrate --fake front 0001_initial
Operations to perform:
  Target specific migration: 0001_initial, from front
Running migrations:
  Applying front.0001_initial... FAKED

events

$ python3 manage.py migrate --plan --fake events 0001_initial
Planned operations:
events.0001_initial
    Create model Event

$ python3 manage.py migrate --fake events 0001_initial
Operations to perform:
  Target specific migration: 0001_initial, from events
Running migrations:
  Applying events.0001_initial... FAKED

lecturers

$ python3 manage.py migrate --plan lecturers
Planned operations:
lecturers.0001_initial
    Create model Course
    Create model Lecturer
    Create model LecturerRating
    Create model Quote
    Create model QuoteVote
    Alter unique_together for quotevote (1 constraint(s))
    Alter unique_together for lecturerrating (1 constraint(s))

Versteh nicht ganz, was da aus den south-Migrationen geworden ist, aber mal probieren!

$ python3 manage.py migrate --fake lecturers 0001_initial
Operations to perform:
  Target specific migration: 0001_initial, from lecturers
Running migrations:
  Applying lecturers.0001_initial... FAKED

documents

$ python3 manage.py migrate --plan documents
Planned operations:
documents.0001_initial
    Create model Document
    Create model DocumentCategory
    Create model DocumentDownload
    Create model DocumentRating
documents.0002_auto_20161124_2111
    Add field user to documentrating
    Add field document to documentdownload
    Add field courses to documentcategory
    Add field lecturers to documentcategory
    Add field category to document
    Add field uploader to document
    Alter unique_together for documentrating (1 constraint(s))
    Alter index_together for documentdownload (1 constraint(s))
documents.0003_auto_20200122_1624
    Alter index_together for documentdownload (1 constraint(s))
    Remove field ip from documentdownload

Vermutlich wurde da erst die initial-Migration via South durchgeführt?

$ python3 manage.py migrate --fake documents 0001_initial
Operations to perform:
  Target specific migration: 0001_initial, from documents
Running migrations:
  Applying documents.0001_initial... FAKED

neue Migrations

$ python3 manage.py migrate front
Operations to perform:
  Apply all migrations: front
Running migrations:
  Applying front.0002_auto_20200119_1707... OK
  Applying front.0003_username_normalization... OK
  Applying front.0004_auto_20200121_1625... OK
  Applying front.0005_remove_user_twitter... OK
  Applying front.0006_change_user_manager... OK

$ python3 manage.py migrate 
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, documents, easy_thumbnails, events, front, lecturers, registration, sessions
Running migrations:
  Applying admin.0001_initial...Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.DuplicateTable: relation "django_admin_log" already exists

$ python3 manage.py migrate --fake-initial admin
Operations to perform:
  Apply all migrations: admin
Running migrations:
  Applying admin.0001_initial... FAKED
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK

$ python3 manage.py migrate 
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, documents, easy_thumbnails, events, front, lecturers, registration, sessions
Running migrations:
  Applying documents.0002_auto_20161124_2111...Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.DuplicateColumn: column "user_id" of relation "documents_documentrating" already exists

Okay, ev. wurde documents.0002_auto_20161124_2111 doch schon gemacht in der bestehenden Datenbank (TODO: verifizieren) - also mal faken:

$ python3 manage.py migrate --fake documents 0002_auto_20161124_2111
Operations to perform:
  Target specific migration: 0002_auto_20161124_2111, from documents
Running migrations:
  Applying documents.0002_auto_20161124_2111... FAKED

aber danach geht immer noch was schief, bei der Migration, die neu ist:

$ python3 manage.py migrate 
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, documents, easy_thumbnails, events, front, lecturers, registration, sessions
Running migrations:
  Applying documents.0003_auto_20200122_1624...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 234, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/operations/models.py", line 530, in database_forwards
    getattr(new_model._meta, self.option_name, set()),
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 379, in alter_index_together
    self._delete_composed_index(model, fields, {'index': True}, self.sql_delete_index)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 397, in _delete_composed_index
    ", ".join(columns),
ValueError: Found wrong number (0) of constraints for documents_documentdownload(document_id, timestamp, ip)

The-Compiler commented 4 years ago

@dbrgn

Ich glaube beim Wechsel von South auf Django Migrations muss man einmal mit --fake migrieren, aber ich habe nicht mehr ganz im Kopf wie das genau läuft...

Einmal komplett mit --fake zu migrieren macht meiner Meinung nach keinen Sinn: Seit der Migration von South zu Django kamen ja noch diverse neue Migrationen dazu, und die sollen ja durchaus ausgeführt werden.

Was ich momentan probiere ist, rauszufinden, welche Migrationen schon von South durchgeführt wurden, und die dann mittels --fake zu faken. Leider gibt's aber nicht zwingend ein 1:1-Mapping von South- zu Django-Migrationen wie's ausschaut, zumindest bei easy_thumbnails.

The-Compiler commented 4 years ago

Momentan stecke ich fest bei documents.0002_auto_20161124_2111. Obwohl die letzten South-Migrationen aus dem 2014 sind und diese Migration aus dem 2016, scheint sie auf dem alten Server grösstenteils schon durchgeführt zu sein.

So wird beispielsweise folgendes dort gemacht:

        migrations.AddField(
            model_name='documentrating',
            name='user',
            field=models.ForeignKey(related_name='DocumentRating', to=settings.AUTH_USER_MODEL, on_delete=django.db.models.deletion.CASCADE),
        ),

und in der Datenbank steht bereits:

CREATE TABLE documents_documentrating (                                         
   id integer NOT NULL,                                                        
   user_id integer NOT NULL,                                                   
   document_id integer NOT NULL,                                               
   rating smallint NOT NULL,                                                   
   CONSTRAINT documents_documentrating_rating_check CHECK ((rating >= 0))                                                                                                                                                                                                                                                                                                            );

ebenso bei:

        migrations.AlterUniqueTogether(
            name='documentrating',
            unique_together=set([('user', 'document')]),
        ),

ALTER TABLE ONLY documents_documentrating                                       
  ADD CONSTRAINT documents_documentrating_user_id_document_id_key UNIQUE (user_id, document_id);

aber der letzte Schritt darin fehlt:

        migrations.AlterIndexTogether(
            name='documentdownload',
            index_together=set([('document', 'timestamp', 'ip')]),
        ),

finde ich nirgends in der Datenbank, weshalb dann auch die darauffolgende Migration failt:

ValueError: Found wrong number (0) of constraints for documents_documentdownload(document_id, timestamp, ip)

@dbrgn hast du da eine Ahnung, was schiefgelaufen sein könnte? Das index_together scheint in https://github.com/studentenportal/web/commit/1264397230d9e3474ad1c1775050afea74e31b5a dazugekommen zu sein (vorher war nur ein "# TODO django 1.5: index_together on document/timestamp/ip" da), aber diese Änderung hat es wohl nie in die Datenbank vom (alten) studentenportal.ch geschafft.

The-Compiler commented 4 years ago

Workaround für Documents-Migration

Wenn ich die Migration fake:

$ python3 manage.py migrate documents 0002 --fake

und dann von Hand den Index hinzufüge (output via sqlmigrate):

CREATE INDEX "documents_documentdownlo_document_id_timestamp_ip_216f63fd_idx" ON "documents_documentdownload" ("document_id", "timestamp", "ip");

dann komme ich einen Schritt weiter:

events

$ python3 manage.py migrate 
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, documents, easy_thumbnails, events, front, lecturers, registration, sessions
Running migrations:
  Applying documents.0003_auto_20200122_1624... OK
  Applying events.0002_event_author...Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.DuplicateColumn: column "author_id" of relation "events_event" already exists

das scheint tatsächlich auch schon vorhanden zu sein, also wieder mal faken:

$ python3 manage.py migrate --fake events 0002_event_author
Operations to perform:
  Target specific migration: 0002_event_author, from events
Running migrations:
  Applying events.0002_event_author... FAKED

registration

$ python3 manage.py migrate 
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, documents, easy_thumbnails, events, front, lecturers, registration, sessions
Running migrations:
  Applying registration.0001_initial...Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.DuplicateTable: relation "registration_registrationprofile" already exists

$ python3 manage.py migrate --fake-initial registration
Operations to perform:
  Apply all migrations: registration
Running migrations:
  Applying registration.0001_initial... FAKED
  Applying registration.0002_registrationprofile_activated... OK
  Applying registration.0003_migrate_activatedstatus... OK
  Applying registration.0004_supervisedregistrationprofile... OK
  Applying registration.0005_activation_key_sha256... OK

sessions

$ python3 manage.py migrate 
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, documents, easy_thumbnails, events, front, lecturers, registration, sessions
Running migrations:
  Applying sessions.0001_initial...Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.DuplicateTable: relation "django_session" already exists

$ python3 manage.py migrate --fake-initial sessions
Operations to perform:
  Apply all migrations: sessions
Running migrations:
  Applying sessions.0001_initial... FAKED

$ python3 manage.py migrate 
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, documents, easy_thumbnails, events, front, lecturers, registration, sessions
Running migrations:
  No migrations to apply.

:tada: :sparkles:

Läuft jetzt mal so auf https://new.studentenportal.ch/ - ich würd aber gerne noch nen Weg finden, der etwas weniger Hacks benötigt. Gibt sicher nochmals nen zweiten Anlauf vor der endgültigen Migration.

~~TODO: Thumbnails scheinen kaputt zu sein!~~ das war nur ich mit deaktiviertem JS

fabianhauser commented 4 years ago

Dokumentendown- und upload sowie Bewertung funktioniert
Zitate hinzufügen und bewerten funktioniert
Bei der Registrierung (getestet mit open@hsr.ch) haut es mir einen 500er um die Ohren:

(Würde aber im Zweifel schonmal Zügeln, das können wir auch nach dem Umzug der daten noch fixen.)

The-Compiler commented 4 years ago

Bei der Registrierung (getestet mit open@hsr.ch) haut es mir einen 500er um die Ohren:

Oh, spannend... Das scheint auch irgendwas mit der Datenmigration zu tun haben, das ging davor nämlich noch.

IntegrityError at /accounts/register/
null value in column "last_login" violates not-null constraint
DETAIL:  Failing row contains (4203, pbkdf2_sha256$[...], null, f, open, , , open@hsr.ch, f, f, 2020-03-09 21:30:55.98991+01, ).

The-Compiler commented 4 years ago

Zum Registrierungs-Fehler, siehe die Django 1.8 release notes:

If you are using a custom user model that inherits from AbstractUser, you’ll need to run makemigrations and generate a migration for your app that contains that model.

Bei uns ist in der initialen Migration schon null=True gesetzt:

('last_login', models.DateTimeField(null=True, verbose_name='last login', blank=True)),

In unserem Datenbankschema ist das aber wohl nicht der Fall....

(Würde aber im Zweifel schonmal Zügeln, das können wir auch nach dem Umzug der daten noch fixen.)

Ich guck mir das heute (beim zweiten Testlauf) noch kurz an. Ich würde doch sehr gerne wissen, wo es noch Inkonsistenzen zwischen Datenbankschema und Django gibt in unseren Daten, bevor dann neue Daten dazukommen. Vermutlich würde Postgres dann schon irgendwie meckern falls nicht, aber diese Probleme sind aus meiner Sicht deutlich einfacher behebbar, wenn man das nicht "in production" macht.

(Oder zumindest will ich wissen, wo es denn Inkonsistenzen gibt, bzw. wie gravierend die sind - aber ab dem Punkt ist ein Fix dann auch nicht mehr weit weg.)

The-Compiler commented 4 years ago

(Oder zumindest will ich wissen, wo es denn Inkonsistenzen gibt, bzw. wie gravierend die sind - aber ab dem Punkt ist ein Fix dann auch nicht mehr weit weg.)

Okay, den Punkt nehm ich zurück - es sind deutlich mehr Änderungen, als gedacht: #19. Ich werde einfach das "not null" während den Migrationen korrigieren, alles andere ist relativ irrelevant (zusätzliche Indices, alte Datenleichen, Datentyp-Änderungen die uns vermutlich nicht betreffen).

The-Compiler commented 4 years ago

Zweiter Anlauf

Diesmal in einer etwas anderen Reihenfolge, um einige der Probleme oben zu vermeiden. Hat geklappt: Es trat nur noch ein (erwarteter) Fehler auf:

easy_thumbnails

Diesmal zuerst, weil es beim letzten Mal Probleme gab damit.

$ python3 manage.py migrate --plan easy_thumbnails
Planned operations:
easy_thumbnails.0001_initial
    Create model Source
    Create model Thumbnail
    Alter unique_together for thumbnail (1 constraint(s))
    Alter unique_together for source (1 constraint(s))
easy_thumbnails.0002_thumbnaildimensions
    Create model ThumbnailDimensions

$ python3 manage.py migrate --fake-initial easy_thumbnails
Operations to perform:
  Apply all migrations: easy_thumbnails
Running migrations:
  Applying easy_thumbnails.0001_initial... FAKED
  Applying easy_thumbnails.0002_thumbnaildimensions... OK

contentypes

$ python3 manage.py migrate --plan contenttypes
Planned operations:
contenttypes.0001_initial
    Create model ContentType
    Alter unique_together for contenttype (1 constraint(s))
contenttypes.0002_remove_content_type_name
    Change Meta options on contenttype
    Alter field name on contenttype
    Raw Python operation
    Remove field name from contenttype

$ python3 manage.py migrate --fake-initial contenttypes
Operations to perform:
  Apply all migrations: contenttypes
Running migrations:
  Applying contenttypes.0001_initial... FAKED
  Applying contenttypes.0002_remove_content_type_name... OK

auth

$ python3 manage.py migrate --plan auth
Planned operations:
auth.0001_initial
    Create model Permission
    Create model Group
    Create model User
auth.0002_alter_permission_name_max_length
    Alter field name on permission
auth.0003_alter_user_email_max_length
    Alter field email on user
auth.0004_alter_user_username_opts
    Alter field username on user
auth.0005_alter_user_last_login_null
    Alter field last_login on user
auth.0006_require_contenttypes_0002
auth.0007_alter_validators_add_error_messages
    Alter field username on user
auth.0008_alter_user_username_max_length
    Alter field username on user
auth.0009_alter_user_last_name_max_length
    Alter field last_name on user
auth.0010_alter_group_name_max_length
    Alter field name on group
auth.0011_update_proxy_permissions
    Raw Python operation ->     Update the content_type of prox…

$ python3 manage.py migrate --fake-initial auth
Operations to perform:
  Apply all migrations: auth
Running migrations:
  Applying auth.0001_initial... FAKED
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK

sessions

$ python3 manage.py migrate --plan sessions
Planned operations:
sessions.0001_initial
    Create model Session

$ python3 manage.py migrate --fake-initial sessions
Operations to perform:
  Apply all migrations: sessions
Running migrations:
  Applying sessions.0001_initial... FAKED

registration

$ python3 manage.py migrate --plan registration
Planned operations:
front.0001_initial
    Create model User
registration.0001_initial
    Create model RegistrationProfile
registration.0002_registrationprofile_activated
    Add field activated to registrationprofile
registration.0003_migrate_activatedstatus
    Raw Python operation
registration.0004_supervisedregistrationprofile
    Create model SupervisedRegistrationProfile
registration.0005_activation_key_sha256
    Alter field activation_key on registrationprofile

$ python3 manage.py migrate --fake-initial registration
Operations to perform:
  Apply all migrations: registration
Running migrations:
  Applying front.0001_initial... FAKED
  Applying registration.0001_initial... FAKED
  Applying registration.0002_registrationprofile_activated... OK
  Applying registration.0003_migrate_activatedstatus... OK
  Applying registration.0004_supervisedregistrationprofile... OK
  Applying registration.0005_activation_key_sha256... OK

(Achtung: hat die initial migration von front gefaked, sollte aber hoffentlich klappen)

front

$ python3 manage.py migrate --fake-initial front
Operations to perform:
  Apply all migrations: front
Running migrations:
  Applying front.0002_auto_20200119_1707... OK
  Applying front.0003_username_normalization... OK
  Applying front.0004_auto_20200121_1625... OK
  Applying front.0005_remove_user_twitter... OK
  Applying front.0006_change_user_manager... OK

events

$ python3 manage.py migrate --plan events
Planned operations:
events.0001_initial
    Create model Event
events.0002_event_author
    Add field author to event

$ python3 manage.py migrate --fake events 0002
Operations to perform:
  Target specific migration: 0002_event_author, from events
Running migrations:
  Applying events.0001_initial... FAKED
  Applying events.0002_event_author... FAKED

(--fake events 0002 statt --fake-initial weil events_event bereits eine Spalte author_id hat)

$ python3 manage.py migrate events
Operations to perform:
  Apply all migrations: events
Running migrations:
  No migrations to apply.

lecturers

$ python3 manage.py migrate --plan lecturers
Planned operations:
lecturers.0001_initial
    Create model Course
    Create model Lecturer
    Create model LecturerRating
    Create model Quote
    Create model QuoteVote
    Alter unique_together for quotevote (1 constraint(s))
    Alter unique_together for lecturerrating (1 constraint(s))

$ python3 manage.py migrate --fake-initial lecturers
Operations to perform:
  Apply all migrations: lecturers
Running migrations:
  Applying lecturers.0001_initial... FAKED

documents

$ python3 manage.py migrate --plan documents
Planned operations:
documents.0001_initial
    Create model Document
    Create model DocumentCategory
    Create model DocumentDownload
    Create model DocumentRating
documents.0002_auto_20161124_2111
    Add field user to documentrating
    Add field document to documentdownload
    Add field courses to documentcategory
    Add field lecturers to documentcategory
    Add field category to document
    Add field uploader to document
    Alter unique_together for documentrating (1 constraint(s))
    Alter index_together for documentdownload (1 constraint(s))
documents.0003_auto_20200122_1624
    Alter index_together for documentdownload (1 constraint(s))
    Remove field ip from documentdownload

$ python3 manage.py migrate --fake documents 0002
Operations to perform:
  Target specific migration: 0002_auto_20161124_2111, from documents
Running migrations:
  Applying documents.0001_initial... FAKED
  Applying documents.0002_auto_20161124_2111... FAKED

Danach wie erwartet ein Fehler:

$ python3 manage.py migrate documents
Operations to perform:
  Apply all migrations: documents
Running migrations:
  Applying documents.0003_auto_20200122_1624...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 234, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/operations/models.py", line 530, in database_forwards
    getattr(new_model._meta, self.option_name, set()),
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 379, in alter_index_together
    self._delete_composed_index(model, fields, {'index': True}, self.sql_delete_index)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 397, in _delete_composed_index
    ", ".join(columns),
ValueError: Found wrong number (0) of constraints for documents_documentdownload(document_id, timestamp, ip)

Fixups von Hand

CREATE INDEX "documents_documentdownlo_document_id_timestamp_ip_216f63fd_idx" ON "documents_documentdownload" ("document_id", "timestamp", "ip");
ALTER TABLE "front_user" ALTER COLUMN last_login DROP NOT NULL;

$ psql -U studentenportal -h postgres studentenportal
Password for user studentenportal: 
psql (11.5 (Debian 11.5-1+deb10u1), server 12.2 (Debian 12.2-2.pgdg100+1))
WARNING: psql major version 11, server major version 12.
         Some psql features might not work.
Type "help" for help.

studentenportal=# CREATE INDEX "documents_documentdownlo_document_id_timestamp_ip_216f63fd_idx" ON "documents_documentdownload" ("document_id", "timestamp", "ip");
CREATE INDEX

studentenportal=# ALTER TABLE "front_user" ALTER COLUMN last_login DROP NOT NULL;
ALTER TABLE

studentenportal=# \q
could not save history to file "/home/studentenportal/.psql_history": No such file or directory

documents 2

$ python3 manage.py migrate documents
Operations to perform:
  Apply all migrations: documents
Running migrations:
  Applying documents.0003_auto_20200122_1624... OK

admin

$ python3 manage.py migrate --plan admin
Planned operations:
admin.0001_initial
    Create model LogEntry
admin.0002_logentry_remove_auto_add
    Alter field action_time on logentry
admin.0003_logentry_add_action_flag_choices
    Alter field action_flag on logentry

$ python3 manage.py migrate --fake-initial admin
Operations to perform:
  Apply all migrations: admin
Running migrations:
  Applying admin.0001_initial... FAKED
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK

$ python3 manage.py migrate --fake-initial admin
Operations to perform:
  Apply all migrations: admin
Running migrations:
  Applying admin.0001_initial... FAKED
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK

done

$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, documents, easy_thumbnails, events, front, lecturers, registration, sessions
Running migrations:
  No migrations to apply.

Sieht also super aus. Registrierung geht jetzt auch (hab's auch einfach mal frech mit open@hsr.ch getestet).

Heisst: Ich bin mir ziemlich sicher, dass ich heute die Migration reibungslos durchführen kann!

The-Compiler commented 4 years ago

Hab noch ein bisschen mit Docker gekämpft (https://github.com/studentenportal/web/issues/248 passierte dann auch auf einmal da) - aber jetzt läuft alles wieder rund, inklusive den neuen Daten. Wheeeeeeee!

studentenportal / deploy

Datenmigration #18

Daten umziehen

Datenbank wiederherstellen

Dump laden

Django-Migration

Migrations-Attempt 1

Auth

front

django_extensions

easy_thumbnails

oauth2

Auth, zum Zweiten

front, zum Zweiten

events

lecturers

documents

neue Migrations

Workaround für Documents-Migration

events

registration

sessions

Zweiter Anlauf

easy_thumbnails

contentypes

auth

sessions

registration

front

events

lecturers

documents

Fixups von Hand

documents 2

admin

done