search

percyliang / sempre

Semantic Parser with Execution
Other
828 stars 301 forks source link

What are the commands to evaluate predictions on the overnight dataset? #227

Closed subendhu19 closed 3 years ago

subendhu19 commented 3 years ago

I have a text file of predicted canonical forms from which I want to compute the denotation accuracy. The canonical forms in this file are predicted by a simplification model that simplifies general utterances into a fixed canonical form so that they can then be parsed by the grammar rules. So now, I need to run the fixed grammar rules to parse the predicted canonical utterances and then execute the logical forms to get the denotation accuracy. What commands should I run for this?

ppasupat commented 3 years ago

Duplicate of #222. Let me put my answer in that thread.