Closed BitcoinNLPer closed 2 years ago
Also curious, as I know, the HumanEval dataset contains 164 problems, according to the latest result in README, the model does not even pass any one of them!
Ya that's correct we discovered issues with the pre-training corpus data. We fixed the issue and released a new pertaining corpus and are in the process of processing the dataset further and pre-training a new GPT-CC.
In the meantime, check out these awesome models: https://huggingface.co/spaces/codeparrot/code-generation-models
This is Codex results
The following result is GPT-CC
Thanks