octabytes / FireO

Google Cloud Firestore modern and simplest convenient ORM package in Python. FireO is specifically designed for the Google's Firestore
https://fireo.octabyte.io
Apache License 2.0
250 stars 29 forks source link

Continuous Pagination #156

Closed SebTota closed 2 years ago

SebTota commented 2 years ago

Hello, I am using pagination to load a list of items and I noticed that the cursor only works on the first time it is used (the second query). After that it just keeps returning the same item. Ex:

@router.get('/items')
def items(show_sold: bool = False, pagination_cursor: str = ''):
    motorcycles = None
    cursor = None
    if pagination_cursor:
        motorcycles = Motorcycle.collection.cursor(pagination_cursor).fetch(1)
        cursor = motorcycles.cursor
    else:
        motorcycles = Motorcycle.collection.fetch(1)
        cursor = motorcycles.cursor
    l_motorcycles = list(motorcycles)
    return MotorcycleListResponse(num_items=len(l_motorcycles),
                                  items=l_motorcycles,
                                  pagination_cursor=cursor)

The first call, /items, to this API route returns item_1 with pagination_cursor='abc123' The second call, /items?pagination_cursor=abc123, returns item_2 with pagination_cursor='abc123' (notice the same cursor hash) The third call, /items?pagination_cursor=abc123, returns the same item_2 with the same cursor id. What is expected is that the db query returns item_3 with (I'm assuming) a new cursor hash.

I'm not really sure what I could be doing wrong here and I would prefer not to have to use the start_at methods because I don't have a specific key I would want to sort the results on. Is what i am trying to accomplish possible with FireO?

SebTota commented 2 years ago

Ended up figuring this out. Casting the iterator to a list works in getting the results, but it doesn't forward the cursor for some reason so you have to do something like l_motorcycles = [m for m in motorcycles]. This is my implementation in the end:

@router.get('/motorcycles', response_model=MotorcycleListResponse)
def get_motorcycles(limit: int = 9, show_sold: bool = False, pagination_cursor: str = None):
    motorcycles_controller = MotorcycleController.collection

    if pagination_cursor:
        motorcycles_controller = motorcycles_controller.cursor(pagination_cursor)
    elif not show_sold:
        motorcycles_controller = motorcycles_controller.filter(sold=False)

    motorcycles_controller = motorcycles_controller.fetch(limit)

    motorcycles = [m.to_dict() for m in motorcycles_controller]
    cursor = motorcycles_controller.cursor

    # I HATE the fact that firestore has no good way to know if there are any results left in your pagination results or a simple count(*) so this is a workaround for that. Granted, it's a costly workaround because all page loads will result in another unnecessary query. 
    if len(motorcycles) < limit or len(list(MotorcycleController.collection.cursor(cursor).fetch(1))) == 0:
        cursor = None

    return MotorcycleListResponse(num_items=len(motorcycles),
                                  items=motorcycles,
                                  pagination_cursor=cursor)