xnuinside / simple-ddl-parser

Simple DDL Parser to parse SQL (HQL, TSQL, AWS Redshift, BigQuery, Snowflake and other dialects) ddl files to json/python dict with full information about columns: types, defaults, primary keys, etc. & table properties, types, domains, etc.
MIT License
179 stars 40 forks source link

SyntaxError: Can't build lexer #199

Closed willshen99 closed 1 year ago

willshen99 commented 1 year ago

Tried to run below sample code:

from simple_ddl_parser import DDLParser

parse_results = DDLParser("""create table dev.data_sync_history(
    data_sync_id bigint not null,
    sync_count bigint not null,
    sync_mark timestamp  not  null,
    sync_start timestamp  not null,
    sync_end timestamp  not null,
    message varchar(2000) null,
    primary key (data_sync_id, sync_start)
); """).run()

print(parse_results)

Received SyntaxError: Can't build lexer

ERROR: /Users/{username}/Library/Python/3.11/lib/python/site-packages/simple_ddl_parser/ddl_parser.py:155: Invalid regular expression for rule 't_AUTOINCREMENT'. global flags not at the start of the expression at position 50
Traceback (most recent call last):
  File "{scriptname}", line 3, in <module>
    parse_results = DDLParser("""create table dev.data_sync_history(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/{username}/Library/Python/3.11/lib/python/site-packages/simple_ddl_parser/parser.py", line 90, in __init__
    self.lexer = lex.lex(object=self, debug=False, debuglog=log)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/{username}/Library/Python/3.11/lib/python/site-packages/ply/lex.py", line 909, in lex
    raise SyntaxError("Can't build lexer")
SyntaxError: Can't build lexer

I have simple-ddl-parser==0.30.0 and ply==3.11 installed

slurpyb commented 1 year ago

I think it has something to do with 3.11 now enforcing global flags at the start of regex expressions. See https://github.com/python/cpython/issues/91222

max-thorpe-sains commented 1 year ago

I also had the very same error when using simple-ddl-parser==0.30.0 and python 3.11.4. Would really love to use this python module, but can't currently.

xnuinside commented 1 year ago

Fix was released in version 0.31.0, thanks everyone for opening the issue & PR with fix!

xnuinside commented 1 year ago

I also added test runner for python 3.11 on CI