search

hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
http://prompttools.readthedocs.io
Apache License 2.0
2.55k stars 216 forks source link

Add OpenAICompatibleChatExperiment class for enhanced LLM compatibility #125

Open agyaatcoder opened 2 months ago

agyaatcoder commented 2 months ago

I have introduced a new class called OpenAICompatibleChatExperiment. This class aims to enhance the compatibility of prompttools with a wider range of large language models (LLMs) that support OpenAI-compatible inference endpoints. This enables usage of popular open source models hosted on Model as API service providers like together.ai, and self hosted models via Huggingface Text Generation Inference/ vLLM/ ollama etc, as they provide openai compatible rest api.

Key features and benefits of the OpenAICompatibleChatExperiment class:

Flexibility: The class allows users to specify the base URL and API key for any OpenAI-compatible inference endpoint, enabling seamless integration with various LLMs. Ease of use: The class maintains a similar interface to the existing OpenAIChatExperiment class, ensuring a smooth transition for users familiar with prompttools. Expanded LLM support: By leveraging OpenAI-compatible endpoints, prompttools can now interface with a broader ecosystem of LLMs, empowering users to explore and utilize different models effortlessly.

CLAassistant commented 2 months ago

CLA assistant check
All committers have signed the CLA.