Open Jeffwan opened 1 year ago
@tloen - We can probably use this issue to track adding an eval harness to the pipeline: https://github.com/EleutherAI/lm-evaluation-harness
I am also instrested in this issue.
Hi is there update?
Hi is there update?
Added initial squad benchmarks, working on piqa.
Also interested
I am just curious whether there's a scientific way to compare the performance between alpaca and alpaca -lora? Does the community have some evaluation scripts to run?