dosubot[bot]
commented
2 months ago To resolve the HTTPError: 400 Client Error: Bad Request for url: http://10.237.119.242:8010/v1/ranking issue when using the Nvidia NIM Reranker in your Python code, ensure that the URL and payload format match the expected format for the on-premises NIM service.
Here is a checklist and example code to help you troubleshoot and resolve the issue:
- Check the URL Format: Ensure the URL is correctly formatted and ends with
/v1/ranking. - Payload Structure: Verify that the payload structure matches the expected format.
- Headers: Ensure the headers are correctly set, especially the
Authorizationheader if required.
Here is an example of how you can structure your request:
import requests
class NVIDIARerank:
def __init__(self, base_url, api_key=None):
self._base_url = base_url
self._api_key = api_key
def rerank(self, query_str, nodes):
session = requests.Session()
headers = {
"Authorization": f"Bearer {self._api_key}",
"Accept": "application/json",
}
payload = {
"model": "nv-rerank-qa-mistral-4b:1",
"query": {"text": query_str},
"passages": [{"text": node} for node in nodes],
}
url = f"{self._base_url}/ranking"
response = session.post(url, headers=headers, json=payload)
response.raise_for_status()
return response.json()
# Example usage
base_url = "http://10.237.119.242:8010/v1"
api_key = "your_api_key" # If required
reranker = NVIDIARerank(base_url, api_key)
query_str = "which way should i go?"
nodes = [
"two roads diverged in a yellow wood, and sorry i could not travel both and be one traveler, long i stood and looked down one as far as i could to where it bent in the undergrowth;",
"then took the other, as just as fair, and having perhaps the better claim because it was grassy and wanted wear, though as for that the passing there had worn them really about the same,",
"and both that morning equally lay in leaves no step had trodden black. oh, i marked the first for another day! yet knowing how way leads on to way i doubted if i should ever come back.",
"i shall be telling this with a sigh somewhere ages and ages hence: two roads diverged in a wood, and i, i took the one less traveled by, and that has made all the difference."
]
try:
result = reranker.rerank(query_str, nodes)
print(result)
except requests.HTTPError as e:
print(f"HTTP error occurred: {e}")Ensure that:
- The
base_urlis correctly set to your on-premises NIM service. - The payload structure matches the expected format.
- The
Authorizationheader is included if your service requires an API key.
If the service works correctly with a curl command, compare the curl request with the Python request to ensure they are identical in terms of URL, headers, and payload.
Additionally, you can refer to the NVIDIA NIMs documentation for more details on setting up and troubleshooting local microservice instances. This documentation provides comprehensive guidance on working with NVIDIA NIMs, both hosted and local instances [1][2].
To continue talking to Dosu, mention @dosu.
vecorro
Bug Description
I'm trying to use a reranker NIM to replace a HiggingFace model. The NIM service works properly from curl. Example:
When I try this reranker, I get
HTTPError: 400 Client Error: Bad Request for url: http://10.237.119.242:8010/v1/rankingVersion
llama-index 0.10.58 / llama-index-postprocessor-nvidia-rerank 0.1.5
Steps to Reproduce
Relevant Logs/Tracbacks