Lite & Super-fast re-ranking for your search & retrieval pipelines. Supports SoTA Listwise and Pairwise reranking based on LLMs and cross-encoders and more. Created by Prithivi Da, open for PRs & Collaborations.
from fastapi import FastAPI
from flashrank import Ranker, RerankRequest
from pydantic import BaseModel
from typing import List
app = FastAPI()
ranker = Ranker()
class Passage(BaseModel):
id: str
text: str
meta: dict
class RankRequest(BaseModel):
query: str
passages: List[Passage]
@app.post("/rank")
async def rank(request: RankRequest):
print(request.query, request.passages)
passages_dict = [passage.model_dump() for passage in request.passages]
rerankrequest = RerankRequest(query=request.query, passages=passages_dict)
result = ranker.rerank(rerankrequest)
return result
App is started with
uvicorn main:app --reload
Traceback (most recent call last):
File "/Users/abcd/.local/share/virtualenvs/flashrank-api-aZfxAaAm/lib/python3.11/site-packages/fastapi/encoders.py", line 322, in jsonable_encoder
data = dict(obj)
^^^^^^^^^
TypeError: 'numpy.float32' object is not iterable
App is started with