hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
http://prompttools.readthedocs.io
Apache License 2.0
2.56k stars 216 forks source link

Added support for MongoDB Export of results for Instruction/Prompt Template #55

Closed pramitbhatia25 closed 11 months ago

pramitbhatia25 commented 11 months ago

Summary of changes: Added export to_mongo_db option after generating results. Added error handling in experiment.py to_mongo_db function. Modified open_ai_chat_completion and open_ai_chat experiments, made them set completion functions only if DEBUG key doesn't exist. Modified load_data function in data_loader.py to return a dict. If there's any error, it is displayed on the UI instead of the app crashing. Added Anthropic / Google PALM / LlamaCpp Completion models in data loader so they don't error out when called. (Support for these still needs work)

Important: Since I've modified data_loader.py / experiment.py, and the playground imports these from the prompttools package, we'll need to release a new version and update it. (I've tested the functions by using them directly in playground.py, they were working fine..)

pramitbhatia25 commented 11 months ago

Hey guys. let me know if I should make any more modifications. I'm working on the to_mongo_db connection for Model Comparison, will need to modify the run_multiple function in data_loader.py file to get it working.

steventkrawczyk commented 11 months ago

This is a great start!

steventkrawczyk commented 11 months ago

One more idea here: can we use streamlit's connectors for mongodb?

https://docs.streamlit.io/knowledge-base/tutorials/databases/mongodb

pramitbhatia25 commented 11 months ago

Gotcha, I'll make these changes, will open another PR when I'm done. Thanks!