issues
search
CarperAI
/
cheese
Used for adaptive human in the loop evaluation of language and embedding models.
MIT License
300
stars
24
forks
source link
Adding the Model to the Loop
#18
Closed
shahbuland
closed
2 years ago
shahbuland
commented
2 years ago
Summary of changes:
Created base model object and added setup for it to communicate with
Modified client and CHEESE object to work with model
Removed some old redundant code
Created a pipeline that does not read from any dataset (i.e. to read from model generations directly)
Update (Sep 12):
Remove init dataset from all pipelines because empty datasets are prone to errors
Now infer dataset from first call to add row to dataset
Tested with architext, though the code for this example will not be present in PR
Summary of changes:
Update (Sep 12):