search

hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
http://prompttools.readthedocs.io
Apache License 2.0
2.56k stars 216 forks source link

Replicate integration #73

Closed steventkrawczyk closed 10 months ago

steventkrawczyk commented 11 months ago

🚀 The feature

Replicate supports serverless inference for LLMs: https://replicate.com/collections/language-models

We should create a replicate experiment.

Motivation, pitch

Easy way to test OSS LLMs, good to compare to Huggingface hosted endpoints

Alternatives

No response

Additional context

No response

NivekT commented 10 months ago

We have integrated with Replicate. You can run various models, such as Llama 2 and Stable Diffusion.

If any model doesn't work with our ReplicateExperiment, please feel free to comment here or open another issue.