eosphoros-ai / sqlgpt-parser

sqlgpt-parser is a Python implementation of an SQL parser that effectively converts SQL statements into Abstract Syntax Trees (AST). By leveraging AST tree comparisons between two SQL queries, it becomes possible to achieve robust evaluation of text-to-SQL models.
Apache License 2.0
25 stars 10 forks source link

OceanBase语法树解析问题 #11

Open Teingi opened 4 months ago

Teingi commented 4 months ago
  1. 不支持replace语句的解析
parse(input=sql, lexer=lexer, debug=debug, tracking=tracking, tokenfunc=tokenfunc)
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/ply/yacc.py", line 333, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/ply/yacc.py", line 1201, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/ply/yacc.py", line 192, in call_errorfunc
    r = errorfunc(token)
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/sqlgpt_parser/parser/oceanbase_parser/parser.py", line 3977, in p_error
    raise err
  File "<string>", line 1
    REPLACE INTO table_name (column1, column2) VALUES ('value1', 'value2')
    ^
SyntaxError: The current version does not support this SQL
>>>
Teingi commented 4 months ago
  1. insert解析只能到表名,不能往下进一步到列级别
>>> from sqlgpt_parser.parser.oceanbase_parser import parser as oceanbase_parser
>>> oceanbase_parser.parse("INSERT INTO table_name VALUES('test1', 'value2')")
Insert(target=Table(name=QualifiedName.of("table_name"), for_update=False))

>>> oceanbase_parser.parse("INSERT INTO table_name (column1, column2) VALUES ('value1', 'value2')")
Insert(target=Table(name=QualifiedName.of("table_name"), for_update=False))
>>>
Teingi commented 4 months ago
  1. 分区表建表语句解析不支持
obclient [test]> CREATE TABLE t1 (tenant_id int, phone_number int(64))   PARTITION BY LIST(tenant_id)        SUBPARTITION BY HASH(phone_number) SUBPARTITIONS 100    (PARTITION p0 VALUES IN(1),     PARTITION p1 VALUES IN(2),     PARTITION p2 VALUES IN(3)    );
Query OK, 0 rows affected (0.363 sec)

obclient [test]>
>>> oceanbase_parser.parse("CREATE TABLE t1 (tenant_id int, phone_number int(64))   PARTITION BY LIST(tenant_id)        SUBPARTITION BY HASH(phone_number) SUBPARTITIONS 100    (PARTITION p0 VALUES IN(1),     PARTITION p1 VALUES IN(2),     PARTITION p2 VALUES IN(3)    )")
Syntax error in input! Parser State:2903 CREATE TABLE identifier LPAREN column_list RPAREN . LexToken(PARTITION,'PARTITION',1,56)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/sqlgpt_parser/parser/oceanbase_parser/parser.py", line 3990, in parse
    return parser.parse(input=sql, lexer=lexer, debug=debug, tracking=tracking, tokenfunc=tokenfunc)
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/ply/yacc.py", line 333, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/ply/yacc.py", line 1201, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/ply/yacc.py", line 192, in call_errorfunc
    r = errorfunc(token)
  File "/home/jingshun.tq/.local/lib/python3.8/site-packages/sqlgpt_parser/parser/oceanbase_parser/parser.py", line 3977, in p_error
    raise err
  File "<string>", line 1
    CREATE TABLE t1 (tenant_id int, phone_number int(64))   PARTITION BY LIST(tenant_id)        SUBPARTITION BY HASH(phone_number) SUBPARTITIONS 100    (PARTITION p0 VALUES IN(1),     PARTITION p1 VALUES IN(2),     PARTITION p2 VALUES IN(3)    )
                                                           ^
SyntaxError: The current version does not support this SQL 56 (PARTITION)
 CREATE TABLE t1 (tenant_id int, phone_number int(64))   PARTITION BY LIST(tenant_id)        SUBPARTITION BY HASH(phone_number) SUBPARTITIONS 100    (PARTITION p0 VALUES IN(1),     PARTITION p1 VALUES IN(2),     PARTITION p2 VALUES IN(3)    )
                                                        ^^^^^^^^^
>>>