Yale-LILY / NLP4Code

Repository for the NLP4Code project at the LILY lab.
Apache License 2.0
5 stars 1 forks source link

Adding more general code generation datasest #33

Open niansong1996 opened 1 year ago

niansong1996 commented 1 year ago

We would like to add more general code generation datasets for evaluation:

Though we already have Python program executors, we still need to adapt to some of the new datasets.

rays1024 commented 1 year ago

For CodeContest's requirement, unable to install bazel on the server. It uses apt install.

rays1024 commented 1 year ago

Also working on preprocessing DS-1000 into jsonl file

niansong1996 commented 1 year ago

Assigning this also to @yilunzhao for CodeContest

niansong1996 commented 1 year ago

We are adding DS-1000 in #44 and HumanEval in #51