search

github / CodeSearchNet

Datasets, tools, and benchmarks for representation learning of code.
https://arxiv.org/abs/1909.09436
MIT License
2.18k stars 385 forks source link

Groundtruth #208

Closed sjmoran closed 4 years ago

sjmoran commented 4 years ago

Is there a plan to release the annotations?

mallamanis commented 4 years ago

Hi Sean,

We don't have a concrete plan for releasing this. As long as we keep the leaderboard active, I'd prefer to keep the data "hidden". We will certainly release the annotations once we archive this project. However, we are still seeing valuable entries to the competition, so I don't expect us to archive this for the next 6 months.

However, if you have a legitimate reason for needing the annotations that would help advance research in some form or another, we'd be happy to share the data with you, under the condition that you won't use it to "cheat" for a leaderboard submission :)