Why are the EXEC scores always zero?

awslabs / gap-text2sql

GAP-text2SQL: Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training

https://arxiv.org/abs/2012.10309

Apache License 2.0

102 stars 25 forks source link

Why are the EXEC scores always zero? #9

Closed JoaoLages closed 3 years ago

JoaoLages commented 3 years ago

For every model I train, including the pretrained model, EXEC scores are zero. does this mean that none of the generated queries matches the gold SQL query when runned to produce an output table?

JoaoLages commented 3 years ago

Seems like the EXEC scores are not calculated on purpose.