replit / ReplitLM

Inference code and configs for the ReplitLM model family

https://huggingface.co/replit

Apache License 2.0

937 stars 83 forks source link

Updating eval.py for updated upstream harness code #26

Closed madhavatreplit closed 1 year ago

madhavatreplit commented 1 year ago

Why

The bigcode-evaluation-harness had some recent updates which break our eval.py's compatibility with the library.

Specifically, they added a few new arguments that are checked for downstream in the script.

We update our script to runnable with the latest version of the harness.

What changed

Updated

evaluation/eval.py: applied our changes over the latest version of main.py from the harness.

Test plan

Ran and reproduced HE number.

Rollout

[x] This is fully backward and forward compatible