Migration Guide

[Jragon]

Hi there,

Saw your post on the django-tenants-schema repo (https://github.com/django-tenants/django-tenants/issues/719#issuecomment-1022484036) and was wondering if you have any info about migrating an existing project over to using RLS from the old schema method.

I'll try and document my process and can report back here if interested. Any info you have lying around would be useful :)

Cheers,

Rory

[xgilest]

Hi Rory we don't have a migration guide, we are a very small team and we want to share this project that has been very useful for us, but it's hard to maintain good documentation.

Anyway, I'll try to detail as much as possible the migration process we followed from django-tenants-schemas to django-tenants-rls.

The first step was adapting the code, this is a very simple step. Most code remains untouched.

Then you'll have to develop an ETL solution, we used https://petl.readthedocs.io/en/stable/. We developed a Django app that connects to the old system and extracts all the data and then inserts it into the new database. The critical part here is maintaining all relational information as the foreign keys ids change. We stored the relation of all the FK on a table and created the objects on the new DB with the references that were in this table.

from django.db import models
from .settings import MAPPING_TABLE_NAME

class M2OKeyMapping(models.Model):
    object_id = models.PositiveIntegerField(null=False)
    external_key = models.CharField(blank=False, max_length=80)
    table_name = models.CharField(blank=False, max_length=256)
    tenant = models.CharField(blank=False, max_length=128)

    class Meta:
        db_table = MAPPING_TABLE_NAME
        unique_together = ('external_key', 'table_name', 'tenant')
        index_together = ('tenant', 'table_name')

The app iterate on every schema and inserts the data on each schema to the new shared table, it adds the schema_id and makes sure all FK and m2m remains consistent. The app we developed is too ad hoc for our case, but please, if you decide to continue with this path please feel free to ask.

The last step is to perform the actual data migration, here is important to notice that you will probably need to users one that is the app user wich cannot bypass rls and one for backups and migrations. Again when you arrive at this point, ask and we'll try to help.

I know there's lots of useful information missing, but I think that this can help you estimate the work to be done.

Cheers Xavi

[Jragon]

Hi Xavi,

Thanks for taking the time to respond. I'm now at the stage where I'm ready to start migrating my data across. Porting my app over to use RLS went relatively smoothly - only a few issues here and there which I will try and document later.

I would love to hear more about your data migration strategy. I haven't done any sort of etl before so would really appreciate seeing how you went about it as an example.

I'll have a go and let you know how it turns out :)

Thanks again,

Rory

[tysonclugg]

I suspect that migration might be achievable using the Django dumpdata and loaddata commands if you have suitable natural keys defined on all your models. Actually achieving this may be non-trivial depending on what you're using in INSTALLED_APPS, but I thought that someone may find this suggestion has some merit regardless.