Closed felixzli closed 2 years ago
In dataset/py150/line_completion/test.json and dataset/javaCorpus/line_completion/test.json, every line's json object has empty "gt": ""
dataset/py150/line_completion/test.json
dataset/javaCorpus/line_completion/test.json
"gt": ""
For example, here is the first line in dataset/py150/line_completion/test.json
{ "id": 51, "input": "<s> import threading <EOL> import IECore <EOL> import Gaffer <EOL> import GafferUI <EOL> import GafferImage <EOL> __all__ = [ ] <EOL> Gaffer . Metadata . registerNode ( <EOL> GafferImage . Display , <EOL> \"<STR_LIT:description>\" , <EOL> \"\"\"<STR_LIT>\"\"\" , <EOL> plugs = { <EOL> \"<STR_LIT:port>\" : [ <EOL> \"<STR_LIT:description>\" , <EOL> \"\"\"<STR_LIT>\"\"\" , <EOL> ] , <EOL> } <EOL> ) <EOL> __plugsPendingUpdate = [ ] <EOL> __plugsPendingUpdateLock = threading . Lock ( ) <EOL> def __scheduleUpdate ( plug , force = False ) : <EOL> if not force : <EOL> global __plugsPendingUpdate <EOL> global __plugsPendingUpdateLock <EOL> with __plugsPendingUpdateLock : <EOL> for p in __plugsPendingUpdate : <EOL> if plug . isSame ( p ) : <EOL> return <EOL> __plugsPendingUpdate . append ( plug ) <EOL> GafferUI . EventLoop . executeOnUIThread ( lambda : __update ( plug ) ) <EOL> def __update ( plug ) : <EOL> node = plug . node ( ) <EOL> if node : <EOL> updateCountPlug = node [ \"<STR_LIT>\" ] <EOL> updateCountPlug . setValue ( updateCountPlug . getValue ( )", "gt": "" }
According to the README.md at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/CodeCompletion-line ,"gt" should not be empty.
"gt"
https://github.com/microsoft/CodeXGLUE/issues/102 ground truth of test set in line completion task is not provided submit predictions to codexglue@microsoft.com to get results
In
dataset/py150/line_completion/test.json
anddataset/javaCorpus/line_completion/test.json
, every line's json object has empty"gt": ""
For example, here is the first line in
dataset/py150/line_completion/test.json
According to the README.md at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/CodeCompletion-line ,
"gt"
should not be empty.