zeno-ml / zeno-build

Build, evaluate, understand, and fix LLM-based apps
MIT License
484 stars 33 forks source link

Refactoring for code cleanliness, parallelization #107

Closed neubig closed 1 year ago

neubig commented 1 year ago

Description

This PR makes several refactoring changes to clean up the pipeline for the chatbot example. The TL;DR is basically that this makes it easier to (1) run zeno-build in parallel on multiple machines, and (2) generate reports from previously finished runs.

There are many fine details such as introducing a locking mechanism to prevent the same experiments from being run twice in parallel, automatically loading the prediction files at the end of the training run, etc.

References

Blocked by

neubig commented 1 year ago

I'll check why the tests are failing tomorrow