Streamlit playground support for localLLMs

hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).

http://prompttools.readthedocs.io

Apache License 2.0

2.54k stars 216 forks source link

Streamlit playground support for localLLMs #57

Open LuvvAggarwal opened 11 months ago

LuvvAggarwal commented 11 months ago

🚀 The feature

Support local llm by connecting streamlit playground with the local dev container User can run the llm locally and provide endpoint to connect LLM with Streamlit or we can provide a wrapper function for LLM to connect with Streamlit

Motivation, pitch

Motivation behind this is Colab where we can connect the notebook with local machine what if we can do same for streamlit playground

Alternatives

No response

Additional context

No response

steventkrawczyk commented 11 months ago

This is an awesome idea by the way! If you implement this we will definitely land it