search

microsoft / CodeXGLUE

CodeXGLUE
MIT License
1.56k stars 366 forks source link

Why does CoSQA test_webquery only has five answers? #112

Closed suiyueliushang closed 2 years ago

suiyueliushang commented 2 years ago

There are 1046 pieces of data in the test_webquery. But why are there only five answers in the webquery_answers? How can I get the left answers? webquery-test-1 1 webquery-test-2 1 webquery-test-3 1 webquery-test-4 0 webquery-test-5 0

celbree commented 2 years ago

We don't release ground truth for this dataset. If you're interested in, please send your submissions to us and we will give you the results.

suiyueliushang commented 2 years ago

I want to apply codeBERT to Chinese and train a Chinese code search model based on CoSQA. However I find there are not enough test data. Thank you very much if you could send the result.

suiyueliushang commented 2 years ago

We don't release ground truth for this dataset. If you're interested in, please send your submissions to us and we will give you the results. Do you mean I seed my predictions to you and you return the test results to me?

celbree commented 2 years ago

Do you mean I seed my predictions to you and you return the test results to me?

Yes. And if you need the ground truth of test set for research use, please also send your request by emails. We cannot share in GitHub.

celbree commented 2 years ago

Could you send a separated email to codexglue@microsoft.com, so that we could send files as attachment to your own email address, not in the GitHub issue (we can't know your email address here..)?