langchain-ai / langchain-nvidia

MIT License
48 stars 15 forks source link

add support for reranking api change w/ truncate parameter #75

Closed mattf closed 1 month ago

mattf commented 1 month ago

rerank models now reject large input by default and provide a truncate parameter to control this behavior.

add optional truncate per -

    truncate: Optional[Literal["NONE", "END"]] = Field(
        description=
            "Truncate input text if it exceeds the model's maximum token length. "
            "Default is model dependent and is likely to raise error if an "
            "input is too long."

usage -

from langchain_nvidia_ai_endpoints import NVIDIARerank

query = "What is acceleration?"
ranker = NVIDIARerank(model=rerank_model, truncate="END")
result = client.compress_documents(documents=large_documents, query=query)