Open boc-tdunn opened 1 month ago
After examining the default django tables that were created up until the failure. Only AUTH_USER has a username field.
Data type Column
Column name schema Data type name Length Scale Nulls
------------------------------- --------- ------------------- ---------- ----- ------
ID SYSIBM INTEGER 4 0 No
PASSWORD SYSIBM VARCHAR 128 0 No
LAST_LOGIN SYSIBM TIMESTAMP 10 6 Yes
IS_SUPERUSER SYSIBM BOOLEAN 1 0 No
USERNAME SYSIBM VARCHAR 30 0 No
FIRST_NAME SYSIBM VARCHAR 30 0 No
LAST_NAME SYSIBM VARCHAR 30 0 No
IS_STAFF SYSIBM BOOLEAN 1 0 No
IS_ACTIVE SYSIBM BOOLEAN 1 0 No
DATE_JOINED SYSIBM TIMESTAMP 10 6 No
EMAIL SYSIBM VARCHAR 254 0 No
... but username is not allowed to be null according to the describe table output. Looking at DJANGO_MIGRATIONS shows the last migration to complete was 0007_alter_validators_add_error_messages. Looking at the next one to run would be 0008_alter_user_username_max_length.py:
from django.contrib.auth import validators
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("auth", "0007_alter_validators_add_error_messages"),
]
operations = [
migrations.AlterField(
model_name="user",
name="username",
field=models.CharField(
error_messages={"unique": "A user with that username already exists."},
help_text=(
"Required. 150 characters or fewer. Letters, digits and @/./+/-/_ "
"only."
),
max_length=150,
unique=True,
validators=[validators.UnicodeUsernameValidator()],
verbose_name="username",
),
),
]
It doesn't appear to alter whether nulls are allowed for the username column.
What's interesting here is that these migration files (the default ones included with django) only seem to be modifying the length of the underlying varchar column. If I modify the migration files directly such as to not change the field length in 0008_alter_user_username_max_length.py and 0010_alter_group_name_max_length.py such that it matches the original length, a no-op, then re-run the migration everything works. Then I undo the changes to the migration files and manually issue ALTER TABLE .. ALTER COLUMN .. SET DATA TYPE VARCHAR(150)
commands to make the columns match. This is a work around for now i suppose.
Any updates? Please let me know if I can clarify or help in anyway.
@boc-tdunn error suggests that column PSUDO_USERNAME defined in migration file has option "null=true". Note: DB2 doesn't allow null values for primary key or unique constraint columns. Hence its better you remove that option. You need to search where exactly column is defined in migrations.
Below are few examples I am giving with fix I had given, Example 1:
class UniqueConstraintProduct(models.Model):
name = models.CharField(max_length=255)
--color = models.CharField(max_length=32, null=True)
++color = models.CharField(max_length=32)
class Meta:
constraints = [
models.UniqueConstraint(fields=["name", "color"], name="name_color_uniq"),
]
Example 2:
class PoolStyle(models.Model):
name = models.CharField(max_length=30)
pool = models.OneToOneField(Pool, models.CASCADE)
another_pool = models.OneToOneField(
-- Pool, models.CASCADE, null=True, related_name="another_style"
++ Pool, models.CASCADE, related_name="another_style"
)
Example 3:
class Celebrity(models.Model):
name = models.CharField("Name", max_length=20)
greatest_fan = models.ForeignKey(
"Fan",
++models.CASCADE,
--models.SET_NULL,
--null=True,
unique=True,
)
Example 4:
class Article(models.Model):
headline = models.CharField(max_length=100)
pub_date = models.DateTimeField()
author = models.ForeignKey(Author, models.SET_NULL, blank=True, null=True)
--slug = models.SlugField(unique=True, blank=True, null=True)
++slug = models.SlugField(unique=True, blank=True)
Example 5:
class Car(models.Model):
--make = models.CharField(max_length=20, unique=True, null=True)
++make = models.CharField(max_length=20, unique=True)
drivers = models.ManyToManyField('Driver', through='CarDriver')
Hi Praveen,
Thanks for having a look! These are the default models that come with a fresh Django install - I haven't modified them. Additionally I'm not seeing anything that makes the field is nullable.
The only thing the migration files are changing with respect to this field is its length. Check out my previous comment:
What's interesting here is that these migration files (the default ones included with django) only seem to be modifying the length of the underlying varchar column. If I modify the migration files directly such as to not change the field length in 0008_alter_user_username_max_length.py and 0010_alter_group_name_max_length.py such that it matches the original length, a no-op, then re-run the migration everything works. Then I undo the changes to the migration files and manually issue
ALTER TABLE .. ALTER COLUMN .. SET DATA TYPE VARCHAR(150)
commands to make the columns match. This is a work around for now i suppose.
So I don't know why a length change is somehow being interpreted as changing the nullability.
@boc-tdunn do you know query for this error ? Can you paste complete query here. you can use "--debug-sql" option to print all queries which gets executed.
So starting with a fresh project I need to run python manage.py migrate
to create tables for the default models provided by django and this is where I see the error described above. That command doesn't take a --debug-sql
option though.
How about this?
py .\manage.py sqlmigrate auth 0008_alter_user_username_max_length
Yields:
Traceback (most recent call last):
File "C:\Users\tdunn\Documents\workspaces\test\manage.py", line 22, in <module>
main()
File "C:\Users\tdunn\Documents\workspaces\test\manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\__init__.py", line 442, in execute_from_command_line
utility.execute()
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\commands\sqlmigrate.py", line 39, in execute
return super().execute(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\base.py", line 459, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\commands\sqlmigrate.py", line 81, in handle
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\db\migrations\loader.py", line 381, in collect_sql
state = migration.apply(state, schema_editor, collect_sql=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\db\migrations\migration.py", line 132, in apply
operation.database_forwards(
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\db\migrations\operations\fields.py", line 241, in database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\ibm_db_django\schemaEditor.py", line 287, in alter_field
self._alter_field(model, old_field, new_field, old_type, new_type,
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\ibm_db_django\schemaEditor.py", line 413, in _alter_field
old_field, new_field = self.alterFieldDataTypeByRemaking(model, old_field, new_field, strict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\ibm_db_django\schemaEditor.py", line 845, in alterFieldDataTypeByRemaking
self.add_field_post(model, tmp_new_field, notnull, p_key, unique)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\ibm_db_django\schemaEditor.py", line 771, in add_field_post
sql = self.sql_create_unique % {'table': self.quote_name(model._meta.db_table), 'name': constraint_name, 'columns': self.quote_name(field.column), 'deferrable': self._deferrable_constraint_sql(None)}
~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
KeyError: 'nulls_distinct'
It looks like its failing to even generate the sql statements.
@boc-tdunn our ibm_db_django supports until Django v 4.2. I see you are using Django 5.1.1. Supporting Django 5.1.1 version is in our pipeline for year 2025.
Try using "--print-sql" option. Something like this, "./manage.py runserver_plus --print-sql"
@boc-tdunn if you can debug, put a break point at ibm_db_dbi.py where error is thrown and see whats sql query passed.
I still don't see --print-sql
or --debug-sql
as options even when switching to Django 4.2. I have two "test" projects that are identically except for django version. I ran this command py .\manage.py sqlmigrate auth 0008_alter_user_username_max_length
on both of them. It is supposed to show you the sql that would be executed if you ran that migraiton.
On Django 4.2 the output is:
--
-- Alter field username on user
--
ALTER TABLE "AUTH_USER" ADD COLUMN "PSUDO_USERNAME" VARCHAR(150);
UPDATE "AUTH_USER" set "PSUDO_USERNAME"="USERNAME";
ALTER TABLE "AUTH_USER" ALTER COLUMN "PSUDO_USERNAME" SET NOT NULL;
ALTER TABLE "AUTH_USER" ADD CONSTRAINT auth_user_psudo_username_f526afb1_uniq UNIQUE ("PSUDO_USERNAME");
ALTER TABLE "AUTH_USER" DROP COLUMN "USERNAME" CASCADE;
ALTER TABLE "AUTH_USER" RENAME COLUMN "PSUDO_USERNAME" TO "USERNAME";
On Django 5:
Traceback (most recent call last):
File "C:\Users\tdunn\Documents\workspaces\test\manage.py", line 22, in <module>
main()
File "C:\Users\tdunn\Documents\workspaces\test\manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\__init__.py", line 442, in execute_from_command_line
utility.execute()
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\commands\sqlmigrate.py", line 39, in execute
return super().execute(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\base.py", line 459, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\core\management\commands\sqlmigrate.py", line 81, in handle
sql_statements = loader.collect_sql(plan)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\db\migrations\loader.py", line 381, in collect_sql
state = migration.apply(state, schema_editor, collect_sql=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\db\migrations\migration.py", line 132, in apply
operation.database_forwards(
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\django\db\migrations\operations\fields.py", line 241, in database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\ibm_db_django\schemaEditor.py", line 287, in alter_field
self._alter_field(model, old_field, new_field, old_type, new_type,
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\ibm_db_django\schemaEditor.py", line 413, in _alter_field
old_field, new_field = self.alterFieldDataTypeByRemaking(model, old_field, new_field, strict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\ibm_db_django\schemaEditor.py", line 845, in alterFieldDataTypeByRemaking
self.add_field_post(model, tmp_new_field, notnull, p_key, unique)
File "C:\Users\tdunn\Documents\workspaces\test\.venv\Lib\site-packages\ibm_db_django\schemaEditor.py", line 771, in add_field_post
sql = self.sql_create_unique % {'table': self.quote_name(model._meta.db_table), 'name': constraint_name, 'columns': self.quote_name(field.column), 'deferrable': self._deferrable_constraint_sql(None)}
~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
So, this does appear to only be an issue with Django 5+.
That said, once I got past this issue using the workaround I described here this library seemed to work ok in Django 5. Do you know when in 2025 Django5 support will be completed? It looks like support for django 4.2 will end April of 2026.
@boc-tdunn Supporting Django 5.0 is in our yr 2025 pipeline. Second half you can expect release.
On a brand new project start I install Django, ibm_db, ibm_db_django. I configure DB like so:
Then run:
Which results in this error:
This is an out-of-the-box django with only a db connection configuration. I'm not sure how to fix this or proceed.
Versions: Django==5.1.1 ibm_db==3.2.3 ibm_db_django==1.5.3.0 Python 3.12.3 DB2/NT64 11.5.9.0