phillxnet
commented
11 months ago Newest Django-oath-toolkit 0001_initial migration contains, in part, the following model creation:
migrations.CreateModel(
name='Application',
fields=[
('id', models.BigAutoField(serialize=False, primary_key=True)),
('client_id', models.CharField(default=oauth2_provider.generators.generate_client_id, unique=True, max_length=100, db_index=True)),
('redirect_uris', models.TextField(help_text='Allowed URIs list, space separated', blank=True)),
('client_type', models.CharField(max_length=32, choices=[('confidential', 'Confidential'), ('public', 'Public')])),
('authorization_grant_type', models.CharField(max_length=32, choices=[('authorization-code', 'Authorization code'), ('implicit', 'Implicit'), ('password', 'Resource owner password-based'), ('client-credentials', 'Client credentials')])),
('client_secret', models.CharField(default=oauth2_provider.generators.generate_client_secret, max_length=255, db_index=True, blank=True)),
('name', models.CharField(max_length=255, blank=True)),
('user', models.ForeignKey(related_name="oauth2_provider_application", blank=True, to=settings.AUTH_USER_MODEL, null=True, on_delete=models.CASCADE)),
('skip_authorization', models.BooleanField(default=False)),
('created', models.DateTimeField(auto_now_add=True)),
('updated', models.DateTimeField(auto_now=True)),
],
options={
'abstract': False,
'swappable': 'OAUTH2_PROVIDER_APPLICATION_MODEL',
},
),Where as we have the same model creation in our storageadmin.sql.in of:
CREATE TABLE oauth2_provider_application (
id integer NOT NULL,
client_id character varying(100) NOT NULL,
user_id integer NOT NULL,
redirect_uris text NOT NULL,
client_type character varying(32) NOT NULL,
authorization_grant_type character varying(32) NOT NULL,
client_secret character varying(255) NOT NULL,
name character varying(255) NOT NULL,
skip_authorization boolean NOT NULL
);
...With a number of follow-ups which if all interim migrations are also then applied, should result in the same state.
Note also that the failure we see in: https://github.com/rockstor/rockstor-core/pull/2727#issuecomment-1785866753 When attempting to apply new Djanog-oath-toolkit migration:
oauth2_provider.0006_alter_application_client_secret
'psycopg2.errors.UndefinedColumn: column oauth2_provider_application.created does not exist'was part of our proposed 'skipped' migrations (on new db instantiation) via the upstream squash commit:
https://github.com/jazzband/django-oauth-toolkit/commit/32bba99bb6bd39f05981430ec3381ad66b096f15
Where "oauth2_provider/migrations/0005_auto_20170514_1141.py" contained our example failure re application.created:
migrations.AddField(
model_name='application',
name='created',
field=models.DateTimeField(auto_now_add=True, default=django.utils.timezone.now),
preserve_default=False,
),
Hooverdan96
As highlighed by @FroggyFlox & @Hooverdan96 as part of #2723, which concerns the smart_manager database and our services setup, we may similarly have a redundancy in how we establish, after dropping storageadmin, a new setup of the default databases "storageadmin".
As Rockstor pre-dates south's integration into Django we have some layers of legacy here. We also have. as part of our DB_SETUP in initrock (for new database instances only), a straight dump of all that has gone before via:
However this dump contains many now redundant, and conflicting, migration details. For example, in this dump/restore file we have the following south migration detail as having been already applied within this sql dump:
And possibly then responsible for the indicated applied state for oauth2_provider's migrations. See the following comment for details: https://github.com/rockstor/rockstor-core/pull/2727#issuecomment-1785866753
Which pertains to an older version of the initial db setup file of oauth2_provider (part of Django-oath-toolkit - an external app). However as detailed in #2710 comment: https://github.com/rockstor/rockstor-core/issues/2710#issuecomment-1779566523 This initial setup/migration file for oauth2_provider has since been re-used as a squashed version now containing many of the follow-up model additions/alterations. Leading, in another part, to this issue's creation, to examine if our initial database setup, consisting of an initial (now old) db snapshot/dump file's restore, should be dropped in favour of using only migrations files to achieve the same. Especially give the dump contains models that are the responsibility of external applications. Inevitably leading to breakage/fragility when those projects modify/squash their migration files.
South migration files, and now Django's also, (over the life time of Rockstor) are 'tagged' as having been applied by their name only via a dedicated db table. This is thought to have now lead to our current inability to re-create a complete/sane oauth2_provider db setup, regarding new db setups only, our supposedly pre-applied (via dump file) migration now contains less than the post squash (into 0001_initial) migrations assume. And so some interim migrations are now missing from our external Django-oauth-toolkit library. But only on fresh db setups, as pre-existing databases prior to our ongoing work in #2727 used versions of this same library prior to this upstream squashing event.