salesforce / WikiSQL

A large annotated semantic parsing corpus for developing natural language interfaces.
BSD 3-Clause "New" or "Revised" License
1.62k stars 322 forks source link

annotate.py throws exception: query word '.' is not in input vocabulary. #13

Closed atulkum closed 6 years ago

atulkum commented 6 years ago

query word "." is not in input vocabulary. ['symsyms', 'symselect', 'symwhere', 'symand', 'symcol', 'symtable', 'symcaption', 'sympage', 'symsection', 'symop', 'symcond', 'symquestion', 'symagg', 'symaggops', 'symcondops', 'symaggops', 'max', 'min', 'count', 'sum', 'avg', 'symcondops', '=', '>', '<', 'op', 'symtable', 'symcol', 'species', 'symcol', 'indole', 'symcol', 'methyl', 'red', 'symcol', 'voges-proskauer', 'symcol', 'citrate', 'symquestion', 'what', 'is', 'the', 'result', 'for', 'salmonella', 'spp.', 'if', 'you', 'use', 'citrate', '?', 'symend'] Traceback (most recent call last): File "annotate.py", line 119, in raise Exception(str(a)) Exception: {'table_id': '1-16083989-1', 'question': {'gloss': ['What', 'is', 'the', 'result', 'for', 'salmonella', 'spp.', 'if', 'you', 'use', 'citrate', '?'], 'words': ['what', 'is', 'the', 'result', 'for', 'salmonella', 'spp.', 'if', 'you', 'use', 'citrate', '?'], 'after': [' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', '', '']}, 'table': {'header': [{'gloss': ['Species'], 'words': ['species'], 'after': ['']}, {'gloss': ['Indole'], 'words': ['indole'], 'after': ['']}, {'gloss': ['Methyl', 'Red'], 'words': ['methyl', 'red'], 'after': [' ', '']}, {'gloss': ['Voges-Proskauer'], 'words': ['voges-proskauer'], 'after': ['']}, {'gloss': ['Citrate'], 'words': ['citrate'], 'after': ['']}]}, 'query': {'sel': 4, 'conds': [[0, 0, {'gloss': ['Salmonella', 'spp', '.'], 'words': ['salmonella', 'spp.', '.'], 'after': [' ', '', '']}]], 'agg': 3}, 'seq_input': {'gloss': ['SYMSYMS', 'SYMSELECT', 'SYMWHERE', 'SYMAND', 'SYMCOL', 'SYMTABLE', 'SYMCAPTION', 'SYMPAGE', 'SYMSECTION', 'SYMOP', 'SYMCOND', 'SYMQUESTION', 'SYMAGG', 'SYMAGGOPS', 'SYMCONDOPS', 'SYMAGGOPS', 'MAX', 'MIN', 'COUNT', 'SUM', 'AVG', 'SYMCONDOPS', '=', '>', '<', 'OP', 'SYMTABLE', 'SYMCOL', 'Species', 'SYMCOL', 'Indole', 'SYMCOL', 'Methyl', 'Red', 'SYMCOL', 'Voges-Proskauer', 'SYMCOL', 'Citrate', 'SYMQUESTION', 'What', 'is', 'the', 'result', 'for', 'salmonella', 'spp.', 'if', 'you', 'use', 'citrate', '?', 'SYMEND'], 'words': ['symsyms', 'symselect', 'symwhere', 'symand', 'symcol', 'symtable', 'symcaption', 'sympage', 'symsection', 'symop', 'symcond', 'symquestion', 'symagg', 'symaggops', 'symcondops', 'symaggops', 'max', 'min', 'count', 'sum', 'avg', 'symcondops', '=', '>', '<', 'op', 'symtable', 'symcol', 'species', 'symcol', 'indole', 'symcol', 'methyl', 'red', 'symcol', 'voges-proskauer', 'symcol', 'citrate', 'symquestion', 'what', 'is', 'the', 'result', 'for', 'salmonella', 'spp.', 'if', 'you', 'use', 'citrate', '?', 'symend'], 'after': [' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', '', ' ', '']}, 'seq_output': {'gloss': ['SYMSELECT', 'SYMAGG', 'COUNT', 'SYMCOL', 'Citrate', 'SYMWHERE', 'SYMCOL', 'Species', 'SYMOP', '=', 'SYMCOND', 'Salmonella', 'spp', '.', 'SYMEND'], 'words': ['symselect', 'symagg', 'count', 'symcol', 'citrate', 'symwhere', 'symcol', 'species', 'symop', '=', 'symcond', 'salmonella', 'spp.', '.', 'symend'], 'after': [' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ', '', ' ', '']}, 'where_output': {'gloss': ['SYMWHERE', 'SYMCOL', 'Species', 'SYMOP', '=', 'SYMCOND', 'Salmonella', 'spp', '.', 'SYMEND'], 'words': ['symwhere', 'symcol', 'species', 'symop', '=', 'symcond', 'salmonella', 'spp.', '.', 'symend'], 'after': [' ', ' ', ' ', ' ', ' ', ' ', ' ', '', ' ', '']}}

vzhong commented 6 years ago

Can you try this with this particular version of coreNLP? I think the new version is not compatible. https://github.com/vzhong/corenlp-docker

vzhong commented 6 years ago

Closing due to lack of activity.

vlad17 commented 6 years ago

I hit this when I downloaded a recent version of CoreNLP as well, and your fix worked -- maybe worth putting on the README?

vzhong commented 6 years ago

Thanks for verifying this @vlad17. I've updated the README.