citusdata / django-multitenant

Python/Django support for distributed multi-tenant databases like Postgres+Citus
MIT License
730 stars 118 forks source link

setval (...seq_id...) fails on distributed tables in loaddata #119

Closed maziar-dandc closed 2 years ago

maziar-dandc commented 2 years ago

sequence_reset_sql needs to be modified to remove FROM "appname_tablename" from the SQL query if the model is a reference/distributed table.

    def sequence_reset_sql(self, style, model_list):
        from django.db import models
        output = []
        qn = self.quote_name
        for model in model_list:
            # Use `coalesce` to set the sequence for each model to the max pk value if there are records,
            # or 1 if there are none. Set the `is_called` property (the third argument to `setval`) to true
            # if there are records (as the max pk value is already in use), otherwise set it to false.
            # Use pg_get_serial_sequence to get the underlying sequence name from the table name
            # and column name (available since PostgreSQL 8)

            for f in model._meta.local_fields:
                if isinstance(f, models.AutoField):
                    output.append(
                        "%s setval(pg_get_serial_sequence('%s','%s'), "
                        "coalesce(max(%s), 1), max(%s) %s null) %s %s;" % (
                            style.SQL_KEYWORD('SELECT'),
                            style.SQL_TABLE(qn(model._meta.db_table)),
                            style.SQL_FIELD(f.column),
                            style.SQL_FIELD(qn(f.column)),
                            style.SQL_FIELD(qn(f.column)),
                            style.SQL_KEYWORD('IS NOT'),
                            style.SQL_KEYWORD('FROM'),
                            style.SQL_TABLE(qn(model._meta.db_table)),
                        )
                    )
                    break  # Only one AutoField is allowed per model, so don't bother continuing.
        return output

taken from: https://github.com/django/django/blob/f5233dce309543c826224be9dfa9c9f4f855f73c/django/db/backends/postgresql/operations.py#L183

If there is a way to easily check if a model is a distributed/reference table, then this can be easily worked around.

maziar-dandc commented 2 years ago

Moved the issue to citus issues board as it doesn't seem to be django's fault, and also, removing the FROM "appname_tablename" didn't help.

Thinking of just undistributing the reference table before loaddata and then redistributing it again afterwards, only used for dev environment so should be ok.