Try to use an async transformer in paginate

[JoJoJoJoJoJoJo]

Hi I found it that there is an overload of async support for apply_items_transformer but it's never called, so would it be a good idea to create an async paginate?

[uriyyo]

Hi @JoJoJoJoJoJoJo,

Async paginate functions already support async transformers.

Example:

from typing import Sequence

from fastapi_pagination.ext.sqlalchemy import paginate
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession

@app.get("/users")
async def get_users(db: AsyncSession = Depends(get_db)) -> Page[UserOut]:
    async def transformer(users: Sequence[User]) -> User:
        for user in users:
            # do something with user
            print(user)

    return await paginate(
        db,
        select(User).order_by(User.created_at),
        transformer=transformer,
    )

[JoJoJoJoJoJoJo]

Thanks for your example! But in my case I want to do a slice first on a list, then execute the query in transformer to avoid query with a very long in (...) condition, maybe it doesn't fit with this way?

[uriyyo]

@JoJoJoJoJoJoJo Could you please show an example?

[JoJoJoJoJoJoJo]

Hi, my case is just like:

@app.get('/category/items')
async def get_category_items(user: User, categories: Sequence[str], db:AsyncSession = Depends(get_db)) -> LimitOffsetPage[UserProperty]:
    async def transformer(categories: Sequence[str], db=db) -> CategoryItem:
        return await query_with_categories_in_db_and_convert_to_Items()

    all_categories = await validate_and_get_sub_categories(categories)
    # then categories can be a large list so we need to do a pagination on it before query in db
    return pagination(all_categories, transformer=transformer)

the function receives parent categories and find all the sub categories with a recursive query, then return related items. So all_categories can be really large, in this case I want to apply the transformer to do the slice after query in db. Or maybe there will be some better practices?

[uriyyo]

Hmmm, That's an interesting case. From where categories come from? Is it from DB or user specify them?

[JoJoJoJoJoJoJo]

Categories are hot keys and barely changed so they are stored in redis and we will find and validate them in validate_and_get_sub_categories according to user's selection.

[uriyyo]

Hi @JoJoJoJoJoJoJo,

New version 0.12.17 has been released. Now you can have async paginate, example:

from fastapi_pagination.async_paginator import paginate

@app.get('/category/items')
async def get_category_items(user: User, categories: Sequence[str], db:AsyncSession = Depends(get_db)) -> LimitOffsetPage[UserProperty]:
    async def transformer(categories: Sequence[str], db=db) -> CategoryItem:
        return await query_with_categories_in_db_and_convert_to_Items()

    all_categories = await validate_and_get_sub_categories(categories)
    # then categories can be a large list so we need to do a pagination on it before query in db
    return await paginate(all_categories, transformer=transformer)

[JoJoJoJoJoJoJo]

Great! Thanks for the updates 👍