search

microsoft / CodeT

MIT License
590 stars 76 forks source link

[RepoCoder] Benchmark test file not found: datasets/random-api-completion.test.jsonl #27

Closed vishwa27yvs closed 5 months ago

vishwa27yvs commented 5 months ago

Hi Authors,

Thank you for the amazing work on RepoCoder!

We wanted to test the performance of just In-file code completion (the "In-file" column in Table 2 (a,b)) - i.e. without any retrieved context, we noticed that the actual benchmark files are present here https://github.com/microsoft/CodeT/blob/35f54d60b152cc31d134b788e702878ad613d9f7/RepoCoder/utils.py#L23 'datasets/random-api-completion.test.jsonl', 'datasets/random-line-completion.test.jsonl'

But we cannot find these files even after unzipping the datasets folder. Could you please upload those files, or suggest if we are doing something wrong here

zfj1998 commented 5 months ago

please refer to this pull request. For permission reasons I can't merge the pull request https://github.com/microsoft/CodeT/pull/20