Aerich breaks after generating migration

[Tears]

After addressing the problem here, I'm addressing it in its own separate issue as well, hoping it might attract more attention.

We have refactored our models from one models.py file into multiple files in a models directory. So we went from

app/
    models.py

to

app/
    models.py/
        core.py
        activities.py

We reflected these changes in Tortoise config. At first, that seemed to go well, as everything was still running smooth. However, after trying to make migrations for new changes, everything breaks. Any aerich command is aborted with the following message:

tortoise.exceptions.ConfigurationError: No model with name 'User' registered in app 'diff_models'.

which was generated by

File "/Users/xxxxx/.local/share/virtualenvs/xxxxxxx-BxfYxR6N/lib/python3.7/site-packages/tortoise/__init__.py", line 119, in         get_related_model
    return cls.apps[related_app_name][related_model_name]
 KeyError: 'User'

(the User model once lived in models.py, but now lives in models/core.py)

This halts all development, as we are not able to generate any migration files anymore. Cloning the project and initiating the database works, but trying to make migrations once again fails. This is the field we're trying to add:

menu_group = fields.ForeignKeyField('models.MenuGroup', on_delete='CASCADE', related_name="menu_items")

We hope the developers of this project can reply as soon as possible as we need to decide wether we're staying with Aerich or having to find something else, because at this moment, we are stuck.

Thank you.

[long2ice]

Drop aerich table in database, rm migrations dir then try init again.

[Tears]

Drop aerich table in database, rm migrations dir then try init again.

Thank you for your fast reply, @long2ice. I've just done the following:

• dropped the aerich table in the database
• deleted the migrations dir
• executed init again

However, this has not seemed to solve the problem. When I run aerich init-db, I get the following message:

tortoise.exceptions.ConfigurationError: Can't create schema due to cyclic fk references

(maybe because all tables except for aerich are still there)

When I run aerich upgrade, I get the following message:

No migrate items

And when trying to make migrations, I get the following message:

AttributeError: 'NoneType' object has no attribute 'pop'

[long2ice]

Maybe because your models struct, which say Can't create schema due to cyclic fk references

[Tears]

Maybe because your models struct, which say Can't create schema due to cyclic fk references

Thank you for your reply, @long2ice. We have not seen this error before, so that is weird (because if it happened earlier in development, we would have gotten this error earlier). Tortoise does not provide us with an explanation, too. Do you have any idea?

[long2ice]

Can you generate db with Tortoise.generate_schema() without aerich?

[Tears]

Can you generate db with Tortoise.generate_schema() without aerich?

Thank you, @long2ice. It does not work there as well, which might suggest that this is a Tortoise ORM issue (which is weird, because the model files have not changed and this is the first time I'm seeing this error). Unfortunately, the error does not state where it thinks the cyclic fk references are.

[long2ice]

Maybe you should check your models

[Tears]

We're checking them right now, but isn't it weird that the models have always worked fine, but now, without any changes to them, we're getting this cyclic fk references-error? What are we looking for in our models that might have caused this error to pop-up so suddenly?

Thanks, @long2ice

[long2ice]

Caused by split to two files?

[Tears]

For example, trying to build the schema using the migration files aerich generated does work correctly (just tested it). Trying to build the schema using Tortoise.generate_schema() fails.

[Tears]

Caused by split to two files?

Thank you, @long2ice. How do we format a foreign key if the model is in another file? For example, when we have models/core.py with a User, how can we reference that model in models/activities.py? Like this:

class Attendance(models.Model):

    id = fields.IntField(pk=True)
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)
    user = fields.ForeignKeyField('models.User', on_delete='CASCADE', related_name="attendees")

?

[long2ice]

Will collect all models content to one file named old_models.py and store in db with each version

[Tears]

Will collect all models content to one file named old_models.py and store in db with each version

Thanks, @long2ice. Not sure if I follow you here. What is the next step we can take to prevent the circular fk references-error and get Aerich up and running again?

[long2ice]

I'm not clear, could you show your all models?

[Tears]

I'm not clear, could you show your all models?

Here they are:

models/core.py

from tortoise import fields, models
from tortoise.contrib.pydantic import pydantic_model_creator
from typing import List
from fastapi import HTTPException, status
from fastapi_permissions import Allow, Deny, Everyone, Authenticated
from datetime import datetime
from tortoise.transactions import atomic

class User(models.Model):
    """
    The User model
    """

    id = fields.IntField(pk=True)
    email = fields.CharField(max_length=250, unique=True, default="blabla@blalba.nl")
    name = fields.CharField(max_length=50, null=True)
    family_name = fields.CharField(max_length=50, null=True)
    category = fields.CharField(max_length=30, default="misc")
    hashed_password = fields.CharField(max_length=128, null=True)
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)
    disabled = fields.BooleanField(default=False)
    roles = fields.ManyToManyField('models.MemberRole', related_name='users')
    groups = fields.ManyToManyField('models.MemberGroup', related_name='users')
    invalidate_permission_cache = fields.BooleanField(default=False)

class MemberRole(models.Model):

    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=250)
    group = fields.ForeignKeyField('models.MemberGroup', on_delete='CASCADE')
    permissions = fields.ManyToManyField('models.MemberPermission', related_name='roles')

class MemberGroup(models.Model):

    id = fields.IntField(pk=True)
    parent = fields.ForeignKeyField('models.MemberGroup', null=True, blank=True, on_delete='RESTRICT')
    name = fields.CharField(max_length=250)
    default_role = fields.ForeignKeyField('models.MemberRole', on_delete='RESTRICT', null=True, blank=True)
    icon_url = fields.CharField(max_length=500, null=True, blank=True)
    description = fields.TextField()
    is_active = fields.BooleanField(default=True)

class MemberPermission(models.Model):

    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=250, unique=True)
    description = fields.TextField()

models/activity_feeds.py

from tortoise import fields, models
from tortoise.contrib.pydantic import pydantic_model_creator
from typing import List
from fastapi import HTTPException, status
from fastapi_permissions import Allow, Deny, Everyone, Authenticated
from datetime import datetime
from tortoise.transactions import atomic

class ActivityFeed(models.Model):

    id = fields.IntField(pk=True)
    parent_group = fields.ForeignKeyField('models.MemberGroup', null=True, blank=True, on_delete='RESTRICT')
    parent_activity = fields.ForeignKeyField('models.Activity', null=True, blank=True, on_delete='RESTRICT')
    is_active = fields.BooleanField(default=True) # set to False to disable feed for group or activity
    allow_comments = fields.BooleanField(default=False) # allow comments or not
    is_archived = fields.BooleanField(default=False) # no one can post, activity is archived
    is_moderated = fields.BooleanField(default=True) # all posts need to be approved
    is_private = fields.BooleanField(default=False) # only members of membergroup or attendees in activity can see feed
    invalidate_acl_cache = fields.BooleanField(default=False)
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)

models/activities.py

from tortoise import fields, models
from tortoise.contrib.pydantic import pydantic_model_creator
from typing import List
from fastapi import HTTPException, status
from fastapi_permissions import Allow, Deny, Everyone, Authenticated
from datetime import datetime
from tortoise.transactions import atomic

class Activity(models.Model):

    id = fields.IntField(pk=True)
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)
    name = fields.CharField(max_length=250)
    description = fields.TextField()
    organizing_group = fields.ForeignKeyField('models.MemberGroup', null=True, blank=True, on_delete='RESTRICT')
    location = fields.CharField(max_length=250)
    max_attendees = fields.IntField(null=True, blank=True)
    signup_deadline = fields.DatetimeField(null=True, blank=True)
    costs = fields.DecimalField(max_digits=4, decimal_places=2, null=True, blank=True)
    start_date = fields.DatetimeField()
    end_date = fields.DatetimeField(null=True, blank=True)
    cover_image_url = fields.CharField(max_length=500)
    hide_attendees = fields.BooleanField(default=False)
    is_closed = fields.BooleanField(default=False) # members can no longer sign up for activity
    is_active = fields.BooleanField(default=True)
    comments_enabled = fields.BooleanField(default=False) # so attendees can attach a comment to their attendance (allergies etc)

class Attendance(models.Model):

    id = fields.IntField(pk=True)
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)
    user = fields.ForeignKeyField('models.User', on_delete='CASCADE', related_name="attendees")
    activity = fields.ForeignKeyField('models.Activity', on_delete='CASCADE')
    comment = fields.TextField(null=True, blank=True)

models/restaurant.py

from tortoise import fields, models
from tortoise.contrib.pydantic import pydantic_model_creator
from typing import List
from fastapi import HTTPException, status
from fastapi_permissions import Allow, Deny, Everyone, Authenticated
from tortoise.transactions import atomic
from datetime import datetime

class MenuGroup(models.Model):
    """
    A MenuGroup is a group of MenuItems, active from a certain date until a certain date.
    You could compare it to the menu cart at a 'real' restaurant. 
    """

    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=300)
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)
    valid_from = fields.DatetimeField()
    valid_till = fields.DatetimeField()
    is_active = fields.BooleanField(default=True)

class ServingShift(models.Model):
    """
    A ServingShift is the time period in which one can order a MenuOrderGroup. This would typically
    be a whole day or a period of a day.
    """

    id = fields.IntField(pk=True)
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)
    starts_at = fields.DatetimeField()
    ends_at = fields.DatetimeField()

class MenuItem(models.Model):

    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=300)
    menu_group = fields.ForeignKeyField('models.MenuGroup', on_delete='CASCADE', related_name="menu_items")
    description = fields.TextField(null=True, blank=True)
    max_available = fields.IntField(null=True, blank=True)
    cover_image_url = fields.CharField(max_length=300, null=True, blank=True)

class MenuOrderGroup(models.Model):

    id = fields.IntField(pk=True)
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)
    user = fields.ForeignKeyField('models.User', on_delete='CASCADE', related_name="orders")
    serving_shift = fields.ForeignKeyField('models.ServingShift', on_delete='RESTRICT', related_name="orders")
    comment = fields.TextField(null=True, blank=True)

class MenuOrder(models.Model):

    id = fields.IntField(pk=True)
    menu_item = fields.ForeignKeyField('models.MenuItem', on_delete='CASCADE', related_name="orders")
    order_group = fields.ForeignKeyField('models.MenuOrderGroup', on_delete='CASCADE', related_name='orders')
    created_at = fields.DatetimeField(auto_now_add=True)
    modified_at = fields.DatetimeField(auto_now=True)
    comment = fields.TextField(null=True, blank=True)

Thank you so much for taking the time to take a look, @long2ice.

[lqmanh]

@long2ice I also have this issue. And no, I have no cyclic FK or anything. My models work fine with previous versions of Aerich but not 0.4.x. Furthermore, Aerich always generate old_models.py file, which is an old behavior (am I right?).

[long2ice]

aerich 0.4.x is a big version and which not compatible with, see changelog for reference.

[lqmanh]

@long2ice It's not about compatibility, I dropped aerich table, removed /migrations and ran aerich init-db again. It worked fine, once, only with that command. Others just won't work.

[long2ice]

See content field in aerich table of database, and have a check, aerich migrations depends that.

[long2ice]

Maybe aerich's bug

[Tears]

Did you see any circular fk references in the models I posted, @long2ice? Or is it indeed a bug in aerich?

Thanks.

[long2ice]

You can try merge all models to one file @Tears

[Tears]

You can try merge all models to one file @Tears

We refactored all our models back to one models file (models/core.py). Unfortunately, we still receive the following error when trying to create a new migration:

tortoise.exceptions.ConfigurationError: No model with name 'User' registered in app 'diff_models'.

Could it be that something is not working correctly within diff_models, @long2ice?

[Tears]

@long2ice Can you report on any progress about understanding or fixing the bugs stated in this issue? If not, we'll sadly have to leave aerich behind us :(

[long2ice]

Sorry I don't have enough time since my busy work and I'm not confirm which is a bug or not, and PR is great welcome.

[FIRDOUS-BHAT]

@long2ice I've ran into the same issue, digged the whole web for the solution but couldn't get anywhere. I've opened an issue as well in aerich github repo.

[mvngne]

I know this is an old problem, but since it's still open and the project seems to be updated from time to time, I wonder if there is a solution to this problem? I am trying to get aerich to work, but as soon as I add foreign keys to my models, aerich init-db or aerich migrate fails.

tortoise.exceptions.ConfigurationError: No app with name 'models' registered. Please check your model names in ForeignKeyFields and configurations

I do have all models in one file called database.py. My config looks like this:

TORTOISE_ORM = {
    "connections": {"default": "mysql://tortoise:tortoise@localhost:3306/sese"},
    "apps": {
        "SelfService": {
            "models": ["models", "aerich.models"],
            "default_connection": "default",
        }
    },
}

from tortoise.models import Model
from tortoise import fields

class Order(Model):
    id = fields.IntField(pk=True)
    ressource = fields.TextField()
    state = fields.ForeignKeyField("models.OrderState", related_name="order")
    created = fields.DatetimeField(auto_now=True)

    def __dict__(self):
        return {"id": self.id, "ressource": self.ressource, "state": self.state}

class OrderState(Model):
    id = fields.IntField(pk=True)
    name = fields.CharField(max_length=255, unique=True)

    class Meta:
        table_description = "Possible order states."

    def __str__(self):
        return self.name