Relation from auto-generated `pydantic_model_creator` is generating `ValidationError` in `tortoise-orm`

[xalien10]

I was trying to use pagination for User model where UserRole has a related relation user_roleswith User model. When I tried to use pydantic_model_creator from tortoise-orm to generate schema for pagination, it is raising pydantic_core._pydantic_core.ValidationError due to relation while model validation.

But if we exclude it explicitly in pydantic_model_creator through exclude, it is able to generate pagination correctly.

Before applying pagination the actual output was following:

[
    {
        "id": "def3cfcd-9232-46cb-a56c-80978e8d7591",
        "created_at": "2023-10-20T12:38:58.605400Z",
        "created_by": null,
        "updated_at": "2023-10-20T12:38:58.605418Z",
        "updated_by": null,
        "email": "ikhtiarcse10ruet@gmail.com",
        "username": null,
        "first_name": null,
        "last_name": null,
        "date_joined": "2023-10-20T12:38:58.605441Z",
        "last_login_at": "2023-10-20T12:38:58.605443Z",
        "is_verified": true,
        "image": null,
        "status": "APPROVAL",
        "user_roles": [
            {
                "id": "3ad93bef-9d07-4b00-9543-89bbd6bdaac7",
                "created_at": "2023-10-20T12:39:01.091174Z",
                "created_by": null,
                "updated_at": "2023-10-20T12:39:01.091184Z",
                "updated_by": null,
                "role": {
                    "id": 2,
                    "created_at": "2023-10-20T12:33:37.375984Z",
                    "created_by": null,
                    "updated_at": "2023-10-20T12:33:37.375987Z",
                    "updated_by": null,
                    "name": "Organization Super Administrator",
                    "type": "DEFAULT",
                    "description": "Default Organization Super Administrator Role",
                    "role_permissions": [
                        {
                            "id": "88c2eafd-19bc-4314-a66c-518acc512aff",
                            "created_at": "2023-10-20T12:33:37.382166Z",
                            "created_by": null,
                            "updated_at": "2023-10-20T12:33:37.382168Z",
                            "updated_by": null
                        },
                        {
                            "id": "146bdf97-1643-4237-a0fe-cbbe4b4b762e",
                            "created_at": "2023-10-20T12:33:37.382160Z",
                            "created_by": null,
                            "updated_at": "2023-10-20T12:33:37.382162Z",
                            "updated_by": null
                        },
                        {
                            "id": "657a7264-c5de-4a08-833f-1d9deb00ca3c",
                            "created_at": "2023-10-20T12:33:37.382155Z",
                            "created_by": null,
                            "updated_at": "2023-10-20T12:33:37.382157Z",
                            "updated_by": null
                        },
                        {
                            "id": "29b93fb3-0a44-4d93-a956-85b4853bb54b",
                            "created_at": "2023-10-20T12:33:37.382150Z",
                            "created_by": null,
                            "updated_at": "2023-10-20T12:33:37.382152Z",
                            "updated_by": null
                        },
                        {
                            "id": "7a6ad2aa-46b7-49a2-863c-d8e45994f76e",
                            "created_at": "2023-10-20T12:33:37.382145Z",
                            "created_by": null,
                            "updated_at": "2023-10-20T12:33:37.382147Z",
                            "updated_by": null
                        },
                        {
                            "id": "44b0f4a9-d66f-40fb-94dd-03cb528a9c21",
                            "created_at": "2023-10-20T12:33:37.382140Z",
                            "created_by": null,
                            "updated_at": "2023-10-20T12:33:37.382142Z",
                            "updated_by": null
                        },
                        {
                            "id": "2f065cd2-313f-4253-bb6e-5ff2db5090ea",
                            "created_at": "2023-10-20T12:33:37.382135Z",
                            "created_by": null,
                            "updated_at": "2023-10-20T12:33:37.382137Z",
                            "updated_by": null
                        },
                        {
                            "id": "f475b9f3-b52e-4aea-ad8d-f2292542ba5d",
                            "created_at": "2023-10-20T12:33:37.382130Z",
                            "created_by": null,
                            "updated_at": "2023-10-20T12:33:37.382132Z",
                            "updated_by": null
                        }
                    ]
                },
                "organization_id": "12ea4a24-c290-47cf-b1a5-e3441208268d",
                "expires_at": null,
                "status": "ACTIVE"
            }
        ]
    }
]

@uriyyo any suggestion about how to achieve this structure using fastapi-pagination and tortoise-orm?

[xalien10]

@uriyyo any updates about this question for underlying related fields data?

[uriyyo]

Hi @xalien10,

I'm sorry for not getting back to you sooner.

Here is code examples that might help you:

from fastapi import FastAPI
from tortoise import Tortoise
from tortoise.contrib.fastapi import register_tortoise
from tortoise.contrib.pydantic import pydantic_model_creator
from tortoise.fields import DatetimeField, ForeignKeyField, IntField, ReverseRelation, TextField
from tortoise.models import Model

from fastapi_pagination import Page, add_pagination
from fastapi_pagination.ext.tortoise import paginate

class User(Model):
    id = IntField(pk=True)
    name = TextField()
    created_at = DatetimeField(auto_now_add=True)

    user_roles: ReverseRelation["UserRole"]

class UserRole(Model):
    id = IntField(pk=True)
    name = TextField()
    created_at = DatetimeField(auto_now_add=True)

    user = ForeignKeyField("models.User", related_name="user_roles")

async def init():
    joe = await User.create(name="Joe")
    jane = await User.create(name="Jane")

    await UserRole.create(name="admin", user=joe)
    await UserRole.create(name="user", user=joe)
    await UserRole.create(name="user", user=jane)

Tortoise.init_models(["__main__"], "models")

UserList = pydantic_model_creator(User, name="UserList")

app = FastAPI()
add_pagination(app)

register_tortoise(
    app,
    generate_schemas=True,
    config={
        "connections": {
            "default": "sqlite://:memory:",
        },
        "apps": {
            "models": {
                "models": ["__main__"],
                "default_connection": "default",
            },
        },
    },
)

app.add_event_handler("startup", init)

@app.get("/users")
async def route() -> Page[UserList]:
    return await paginate(User, prefetch_related=True)

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app)

My suggestions for where the problem could be on your side:

1. You forget to add Tortoise.init_models before calling pydantic_model_creator. Please, take a look here - https://tortoise.github.io/contrib/pydantic.html#pydanticmeta-callables
2. You need to add prefetch_related when paginate queries with relationship.

[xalien10]

@uriyyo Instead of model User, can we use queryset? like this one -

@app.get("/users")
async def route() -> Page[UserList]:
    return await paginate(await User.filter(name="John Snow"), prefetch_related=True)

[uriyyo]

@xalien10 Sure

from fastapi_pagination.ext.tortoise import paginate

@app.get("/users")
async def route() -> Page[UserList]:
    return await paginate(User.filter(name="Joe"), prefetch_related=True)

[xalien10]

@uriyyo do we need to explicitly write the reverse relation like user_roles in User model?

as you can see here model_validate failing. And in the items, it can get the queryset properly. So, I'm wondering what could go wrong? I've added early init in the schema before creating any Pydantic model from tortoise-orm

[uriyyo]

do we need to explicitly write the reverse relation like user_roles in User model?

Yup, you need to do it

[uriyyo]

@xalien10 Any updates? Can I close this issue?

[xalien10]

Actually, I tried to use this approach but it didn't help with my service layer with repository pattern. I'm extremely sorry that I couldn't report it earlier than now

[uriyyo]

Is there anything I can help you with?