search

microsoft / CodeXGLUE

CodeXGLUE
MIT License
1.51k stars 363 forks source link

Can I use the test set of the Code-Completion-token task as the answer? #106

Closed GoneZ5 closed 2 years ago

GoneZ5 commented 2 years ago

Dear author,

I can not find the py150 answers.txt of the Code-Completion-token task, can I directly use the test set as the answers? Or submit my predictions to codexglue@microsoft.com?

The same problem appears in the task of the Code-Completion-line and text to code generation. Should I submit all predictions of them to you?

Looking forward to your reply.

Best wish

celbree commented 2 years ago

For the Code-Completion-token task, you can directly use the test set as the answer because the next token is just in the same file. While for the Code-Completion-line task, we don't provide the ground truth of test set. You can submit your results to us.

GoneZ5 commented 2 years ago

For the Code-Completion-token task, you can directly use the test set as the answer because the next token is just in the same file. While for the Code-Completion-line task, we don't provide the ground truth of test set. You can submit your results to us.

Thanks for your reply!