shimizukawa commented 2 years ago

Feature or Bugfix

Feature and Bugfix

Purpose

to support manage.py migrate for django auth, contenttypes, and etc.

Detail

Incompatible changes

To specify SORTKEY for Redshift, you must use django_redshift_backend.SortKey for Model.Meta.ordering instead of bearer string.
django_redshift_backend.distkey.DistKey is moved to django_redshift_backend.DistKey. However old name is still supported for a compatibility.
Now django-redshift-backend doesn't support can_rollback_ddl. Originally, Redshift did not support column name/type(size) changes within a transaction. Please refer https://github.com/jazzband/django-redshift-backend/issues/96
changed the behavior of implicit not null column addition. previously, adding a not null column was implicitly changed to allow null. now adding not null without default raises a programmingerror exception.

Bug Fixes:

37: fix Django contenttype migration that cause `ProgrammingError: cannot drop sortkey column

"name"` exception.
64: fix Django auth migration that cause `NotSupportedError: column "content_type__app_label"

specified as distkey/sortkey is not in the table "auth_permission"` exception.

Features:

63 Support changing a field from NOT NULL to NULL on migrate / sqlmigrate.
Support VARCHAR size changing for UNIQUE, PRIMARY KEY, FOREIGN KEY.
Support backward migration for DROP NOT NULL column wituout DEFAULT. One limitation is that the DEFAULT value is set to match the type. This is because the only way for Redshift to add NOT NULL without default is to recreate the table.

Relates

37
64
63
96

codecov[bot] commented 2 years ago

Codecov Report

Merging #97 (63a4921) into master (392a096) will increase coverage by 10.55%. The diff coverage is 56.18%.

@@             Coverage Diff             @@
##           master      #97       +/-   ##
===========================================
+ Coverage   52.44%   62.99%   +10.55%     
===========================================
  Files           3        4        +1     
  Lines         286      427      +141     
  Branches       80      122       +42     
===========================================
+ Hits          150      269      +119     
+ Misses        125      116        -9     
- Partials       11       42       +31

Impacted Files	Coverage Δ
django_redshift_backend/distkey.py	`0.00% <0.00%> (-100.00%)`	:arrow_down:
django_redshift_backend/base.py	`62.12% <56.00%> (+12.49%)`	:arrow_up:
django_redshift_backend/meta.py	`58.82% <58.82%> (ø)`
django_redshift_backend/__init__.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 392a096...63a4921. Read the comment docs.

shimizukawa commented 2 years ago

django migrations with django-3.2.12

(.venv) @shimizukawa ➜ /workspaces/django-redshift-backend/examples/proj1 (fix/django-migrartions ✗) $ python manage.py --version
3.2.12

(.venv) @shimizukawa ➜ /workspaces/django-redshift-backend/examples/proj1 (fix/django-migrartions ✗) $ python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions, testapp
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying sessions.0001_initial... OK
  Applying testapp.0001_initial... OK

(.venv) @shimizukawa ➜ /workspaces/django-redshift-backend/examples/proj1 (fix/django-migrartions ✗) $ python manage.py showmigrations
admin
 [X] 0001_initial
 [X] 0002_logentry_remove_auto_add
 [X] 0003_logentry_add_action_flag_choices
auth
 [X] 0001_initial
 [X] 0002_alter_permission_name_max_length
 [X] 0003_alter_user_email_max_length
 [X] 0004_alter_user_username_opts
 [X] 0005_alter_user_last_login_null
 [X] 0006_require_contenttypes_0002
 [X] 0007_alter_validators_add_error_messages
 [X] 0008_alter_user_username_max_length
 [X] 0009_alter_user_last_name_max_length
 [X] 0010_alter_group_name_max_length
 [X] 0011_update_proxy_permissions
 [X] 0012_alter_user_first_name_max_length
contenttypes
 [X] 0001_initial
 [X] 0002_remove_content_type_name
sessions
 [X] 0001_initial
testapp
 [X] 0001_initial

(.venv) @shimizukawa ➜ /workspaces/django-redshift-backend/examples/proj1 (fix/django-migrartions ✗) $ python manage.py migrate contenttypes zero
Operations to perform:
  Unapply all migrations: contenttypes
Running migrations:
  Rendering model states... DONE
  Unapplying auth.0012_alter_user_first_name_max_length... OK
  Unapplying auth.0011_update_proxy_permissions... OK
  Unapplying auth.0010_alter_group_name_max_length... OK
  Unapplying auth.0009_alter_user_last_name_max_length... OK
  Unapplying auth.0008_alter_user_username_max_length... OK
  Unapplying auth.0007_alter_validators_add_error_messages... OK
  Unapplying auth.0006_require_contenttypes_0002... OK
  Unapplying contenttypes.0002_remove_content_type_name... OK
  Unapplying auth.0005_alter_user_last_login_null... OK
  Unapplying auth.0004_alter_user_username_opts... OK
  Unapplying auth.0003_alter_user_email_max_length... OK
  Unapplying auth.0002_alter_permission_name_max_length... OK
  Unapplying admin.0003_logentry_add_action_flag_choices... OK
  Unapplying admin.0002_logentry_remove_auto_add... OK
  Unapplying admin.0001_initial... OK
  Unapplying auth.0001_initial... OK
  Unapplying contenttypes.0001_initial... OK
(.venv) @shimizukawa ➜ /workspaces/django-redshift-backend/examples/proj1 (fix/django-migrartions ✗) $ python manage.py migrate sessions zero
Operations to perform:
  Unapply all migrations: sessions
Running migrations:
  Rendering model states... DONE
  Unapplying sessions.0001_initial... OK
(.venv) @shimizukawa ➜ /workspaces/django-redshift-backend/examples/proj1 (fix/django-migrartions ✗) $ python manage.py migrate testapp zero
Operations to perform:
  Unapply all migrations: testapp
Running migrations:
  Rendering model states... DONE
  Unapplying testapp.0001_initial... OK
(.venv) @shimizukawa ➜ /workspaces/django-redshift-backend/examples/proj1 (fix/django-migrartions ✗) $ python manage.py showmigrations
admin
 [ ] 0001_initial
 [ ] 0002_logentry_remove_auto_add
 [ ] 0003_logentry_add_action_flag_choices
auth
 [ ] 0001_initial
 [ ] 0002_alter_permission_name_max_length
 [ ] 0003_alter_user_email_max_length
 [ ] 0004_alter_user_username_opts
 [ ] 0005_alter_user_last_login_null
 [ ] 0006_require_contenttypes_0002
 [ ] 0007_alter_validators_add_error_messages
 [ ] 0008_alter_user_username_max_length
 [ ] 0009_alter_user_last_name_max_length
 [ ] 0010_alter_group_name_max_length
 [ ] 0011_update_proxy_permissions
 [ ] 0012_alter_user_first_name_max_length
contenttypes
 [ ] 0001_initial
 [ ] 0002_remove_content_type_name
sessions
 [ ] 0001_initial
testapp
 [ ] 0001_initial

(.venv) @shimizukawa ➜ /workspaces/django-redshift-backend/examples/proj1 (fix/django-migrartions ✗) $ python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions, testapp
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying sessions.0001_initial... OK
  Applying testapp.0001_initial... OK

shimizukawa commented 2 years ago

Glad to see Django migration working! It was worth spending 11 days of my personal time for this fix.

jazzband / django-redshift-backend

bug fixes and support several features for django migrations #97

Feature or Bugfix

Purpose

Detail

Incompatible changes

Bug Fixes:

37: fix Django `contenttype` migration that cause `ProgrammingError: cannot drop sortkey column

64: fix Django `auth` migration that cause `NotSupportedError: column "content_type__app_label"

Features:

63 Support changing a field from NOT NULL to NULL on migrate / sqlmigrate.

Relates

37

64

63

96

Codecov Report

jazzband / django-redshift-backend

bug fixes and support several features for django migrations #97

Feature or Bugfix

Purpose

Detail

Incompatible changes

Bug Fixes:

37: fix Django contenttype migration that cause `ProgrammingError: cannot drop sortkey column

64: fix Django auth migration that cause `NotSupportedError: column "content_type__app_label"

Features:

63 Support changing a field from NOT NULL to NULL on migrate / sqlmigrate.

Relates

37

64

63

96

Codecov Report

37: fix Django `contenttype` migration that cause `ProgrammingError: cannot drop sortkey column

64: fix Django `auth` migration that cause `NotSupportedError: column "content_type__app_label"