A central, open resource for data and tools related to chain-of-thought reasoning in large language models. Developed @ Samwald research group: https://samwald.info/
MIT License
863
stars
69
forks
source link
Annotator "star" feature does not work properly #127
The feature with the star (key: "preferred") does not really work . (The primary I think the primary problem is, that it saves values as bool, when the have to be saved as string. Apart from the minor problem that you cannot "unclick" a star, if you clicked it)
The error is that you cannot load the data back into ThoughtSource (the "normal" cot lib we always use) after annotating it.
If you already have that problem here is the quick fix:
You can replace the values in the json before loading it in ThoughtSource:
replace
"value": false
by
"value": "false"
and
"value": true
by
"value": "true"
Then you can load it. The annotator can afterwards still read the file too, but the star annotation is not shown.
The feature with the star (key: "preferred") does not really work . (The primary I think the primary problem is, that it saves values as bool, when the have to be saved as string. Apart from the minor problem that you cannot "unclick" a star, if you clicked it)
The error is that you cannot load the data back into ThoughtSource (the "normal" cot lib we always use) after annotating it.
If you already have that problem here is the quick fix: You can replace the values in the json before loading it in ThoughtSource: replace "value": false by "value": "false" and "value": true by "value": "true"
Then you can load it. The annotator can afterwards still read the file too, but the star annotation is not shown.