[Bug]: 401 using `watsonx` provider

rohanarora commented 1 week ago

What happened?

Attempted to leverage the watsonx provider as opposed to the watsonx_text provider
Renamed max_new_tokens parameter to max_tokens as required
Request to the watsonx.ai back-end returning 401 as stated in the relevant log output below

Relevant log output

Traceback (most recent call last):
  File "/Users/rohanarora/development/lumyn/.venv/lib/python3.12/site-packages/litellm/llms/openai_like/chat/handler.py", line 353, in completion
    response = client.post(
               ^^^^^^^^^^^^
  File "/Users/rohanarora/development/lumyn/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 389, in post
    raise e
  File "/Users/rohanarora/development/lumyn/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 375, in post
    response.raise_for_status()
  File "/Users/rohanarora/development/lumyn/.venv/lib/python3.12/site-packages/httpx/_models.py", line 763, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://us-south.ml.cloud.ibm.com/ml/v1/text/chat?version=2024-03-13'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/rohanarora/development/lumyn/.venv/lib/python3.12/site-packages/litellm/main.py", line 2626, in completion
    response = watsonx_chat_completion.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rohanarora/development/lumyn/.venv/lib/python3.12/site-packages/litellm/llms/watsonx/chat/handler.py", line 120, in completion
    return super().completion(
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/rohanarora/development/lumyn/.venv/lib/python3.12/site-packages/litellm/llms/openai_like/chat/handler.py", line 360, in completion
    raise OpenAILikeError(
litellm.llms.openai_like.common_utils.OpenAILikeError: {"errors":[{"code":"authentication_token_not_valid","message":"Failed to authenticate the request due to invalid token: Failed to parse and verify token","more_info":"https://cloud.ibm.com/apidocs/watsonx-ai"}],"trace":"cbd8b449ceaa5e74ae3369a89e408ea5","status_code":401}



### Are you a ML Ops Team?

No

### What LiteLLM version are you on ?

v1.52.8

### Twitter / LinkedIn details

_No response_

rohanarora commented 1 week ago

Primarily is it that instead of the api_key here we should be leveraging:

api_params['token']

as it is the token which is needed for authentication here as opposed to the API key at this stage?

Happy to make a PR if this is an acceptable solution.