Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
We want to test across different infrastructure providers. Today, our only way to test OSS models is (1) locally or (2) from huggingface inference APIs. We want to add huggingface inference endpoints as well.
🚀 The feature
Today we support huggingface inference API, which are lightweight instances for testing models.
There is a different feature, called inference endpoints which is for larger models or more persistent instances. See https://huggingface.co/inference-endpoints
Motivation, pitch
We want to test across different infrastructure providers. Today, our only way to test OSS models is (1) locally or (2) from huggingface inference APIs. We want to add huggingface inference endpoints as well.
Alternatives
N/A
Additional context
N/A