antlr / grammars-v4

Grammars written for ANTLR v4; expectation that the grammars are free of actions.
MIT License
10.16k stars 3.7k forks source link

[SQLite] Unexpected errors with legit SELECT statement using `parse` rule #2640

Open ryangawei opened 2 years ago

ryangawei commented 2 years ago

Hi there,

I'm using the SQLite grammar,

def parse(query: str):
    input_stream = InputStream(query)
    lexer = SQLiteLexer(input_stream)
    stream = CommonTokenStream(lexer)
    parser = SQLiteParser(stream)

    # Weird errors happen when it starts with `parse()`
    # tree = parser.parse()
    tree = parser.select_stmt()

    return tree

if I start with the rule parse, I get errors using the following queries,

>>> SELECT count(*) FROM postseason WHERE YEAR  =  1885 AND ties  =  1;
line 1: 52 extraneous input 'AND' expecting {<EOF>, ';', 'ALTER', 'ANALYZE', 'ATTACH', 'BEGIN', 'COMMIT', 'CREATE', 'DEFAULT', 'DELETE', 'DETACH', 'DROP', 'END', 'EXPLAIN', 'INSERT', 'PRAGMA', 'REINDEX', 'RELEASE', 'REPLACE', 'ROLLBACK', 'SAVEPOINT', 'SELECT', 'UPDATE', 'VACUUM', 'VALUES', 'WITH'}

>>> SELECT document_name FROM documents GROUP BY document_type_code ORDER BY count(*) DESC LIMIT 3 INTERSECT SELECT document_name FROM documents GROUP BY document_structure_code ORDER BY count(*) DESC LIMIT 3
line 1: 95 extraneous input 'INTERSECT' expecting {<EOF>, ';', 'ALTER', 'ANALYZE', 'ATTACH', 'BEGIN', 'COMMIT', 'CREATE', 'DEFAULT', 'DELETE', 'DETACH', 'DROP', 'END', 'EXPLAIN', 'INSERT', 'PRAGMA', 'REINDEX', 'RELEASE', 'REPLACE', 'ROLLBACK', 'SAVEPOINT', 'SELECT', 'UPDATE', 'VACUUM', 'VALUES', 'WITH'}

And if I switch to select_stmt, the errors are gone. And it seems to me that the two SQL queries have correct syntax. What's happening here?

kaby76 commented 2 years ago

Verified. keyword is missing TIES_ keyword. This was introduced here with, of course, no explanation. Assuming the addition of the keyword was correct (doubtful), it should be added to the "keyword" production, and all other "keywords" added in that commit gone through.