zeno-ml / zeno-build

Build, evaluate, understand, and fix LLM-based apps
MIT License
482 stars 33 forks source link

Add code generation analysis #130

Closed zorazrw closed 1 year ago

zorazrw commented 1 year ago

Analysis of Code Generation

This PR adds an analysis for code generation, supporting:

Blocked by

neubig commented 1 year ago

OK, after much poking around this is working for the basic models! Here are some scores on odex for codegen and gpt2:

visualization

I will merge this branch in and we can continuously improve in future PRs. Thanks for the contribution @zorazrw !