search

openai / human-eval

Code for the paper "Evaluating Large Language Models Trained on Code"
MIT License
2.31k stars 330 forks source link

Will this be helpful for people reading the paper ? #1

Open jalotra opened 3 years ago

jalotra commented 3 years ago

I was reading your paper. And was interested in seeing what the test data looks like. But you have given it in .jsonl format which is okay if you are doing a careful analysis of the test data.

But for quick readers parsing the test data requires willingness and they might not want to do it.

So I parsed the data.jsonl into Human Readable Prompt and Canoncial Sections : https://gist.github.com/jalotra/f4d119e8108fcd44b932103d73140381.

Will this add value to this repo if we can add this for a quick read ?

geajack commented 1 year ago

I agree, two or three example datapoints in the README for this repo would be really helpful.

HenryHZY commented 1 year ago

Great sharing!