zeno-ml / zeno-build

Build, evaluate, understand, and fix LLM-based apps

MIT License

482 stars 33 forks source link

Make chatbot example configurable #161

Closed neubig closed 1 year ago

neubig commented 1 year ago

Description

This PR organizes the configuration file for the chatbot example into sections, and makes several elements configurable through command line options:

Which experiments to run (i.e. what parameters to sweep over)
What models to use
What prompts to use
What inference method to use for open-source models (huggingface or vllm)

It also makes a few related changes

Organizes imported libraries to make openai required (since it's pretty standard anyway), and remove the requirement for cohere if cohere models are not used.

I have re-run experiments using this and things seem to be working OK.

References

Fixes https://github.com/zeno-ml/zeno-build/issues/160

Blocked by

NA

neubig commented 1 year ago

Ah, good catch. I'll do this. It'll take a little while so let me merge this PR and I'll improve the doc in a new PR. Created an issue here: https://github.com/zeno-ml/zeno-build/issues/163