[Bug]: LlamaIndex call OpenAI Like Api got err : NotFoundError 404

[ezscode]

What happened?

I config zhipu api in config.yaml and call successfully with client.chat.completions.create

but I try to call the proxy in llamaindex with OpenAILike : https://docs.llamaindex.ai/en/stable/api_reference/llms/openai_like/

model_name = 'zhipu--glm-4' 
litellm_key = "sk-1234"
litellm_base_url = 'http://localhost:4000/'
llm = OpenAILike(model=model_name, api_base=litellm_base_url, api_key=litellm_key)
llm.complete("Hello World!")

litellm server error:

litellm.exceptions.NotFoundError: litellm.NotFoundError: NotFoundError: OpenAIException - Error code: 404 - {'timestamp': '2024-09-19T06:56:27.494+00:00', 'status': 404, 'error': 'Not Found', 'path': '/v4/completions'}
Received Model Group=zhipu--glm-4
Available Model Group Fallbacks=None

what can I do to fix it ?

---

PS : the recommand method to use AzureOpenAI is out-dated

Relevant log output

litellm proxy server

  File "/Users/pc087/miniconda3/lib/python3.11/site-packages/litellm/utils.py", line 8189, in exception_type
    raise e
  File "/Users/pc087/miniconda3/lib/python3.11/site-packages/litellm/utils.py", line 6531, in exception_type
    raise NotFoundError(
litellm.exceptions.NotFoundError: litellm.NotFoundError: NotFoundError: OpenAIException - Error code: 404 - {'timestamp': '2024-09-19T06:56:27.494+00:00', 'status': 404, 'error': 'Not Found', 'path': '/v4/completions'}
Received Model Group=zhipu--glm-4
Available Model Group Fallbacks=None

***
llama-index endpont 

File "/Users/pc087/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 937, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/Users/pc087/miniconda3/lib/python3.11/site-packages/openai/_base_client.py", line 1041, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'message': "litellm.NotFoundError: NotFoundError: OpenAIException - Error code: 404 - {'timestamp': '2024-09-19T06:56:27.494+00:00', 'status': 404, 'error': 'Not Found', 'path': '/v4/completions'}\nReceived Model Group=zhipu--glm-4\nAvailable Model Group Fallbacks=None", 'type': None, 'param': None, 'code': '404'}}

Twitter / LinkedIn details

No response

[ezscode]

I find the best way to integrate litellm in llama-index:

https://docs.llamaindex.ai/en/stable/examples/llm/litellm/