Closed ishaan-jaff closed 6 months ago
@waleedkadous @kylehh I thought litellm would be useful here to make benchmarking easier
noticed you have an outstanding PR for adding HF inference (which litellm supports already)
LLMperf v2 released and LiteLLM was added as one of the LLM client Close this PR
This PR adds support for the above mentioned LLMs using LiteLLM https://github.com/BerriAI/litellm/ LiteLLM is a lightweight package to simplify LLM API calls - use any llm as a drop in replacement for gpt-3.5-turbo.
Example