rerank models now reject large input by default and provide a truncate parameter to control this behavior.
add optional truncate per -
truncate: Optional[Literal["NONE", "END"]] = Field(
description=
"Truncate input text if it exceeds the model's maximum token length. "
"Default is model dependent and is likely to raise error if an "
"input is too long."
usage -
from langchain_nvidia_ai_endpoints import NVIDIARerank
query = "What is acceleration?"
ranker = NVIDIARerank(model=rerank_model, truncate="END")
result = client.compress_documents(documents=large_documents, query=query)
rerank models now reject large input by default and provide a truncate parameter to control this behavior.
add optional
truncate
per -usage -