apache / datafusion-sqlparser-rs

Extensible SQL Lexer and Parser for Rust
Apache License 2.0
2.81k stars 543 forks source link

Expected end of statement, found: COMMENT while parsing MySQL `CREATE TABLE` statement #991

Open gadgetlabs opened 1 year ago

gadgetlabs commented 1 year ago

Hi,

I'm using sqlparser = "0.38.0", trying to parse the below with

let result = Parser::parse_sql(dialect.as_ref(), &sql);
    let ast: Vec<Statement> = Vec::new();

    match result {
        Ok(value) => {
            println!("Parse Ok");
            let ast = value;
        },
        Err(err) => {
            eprintln!("Error: {}", err);
            std::process::exit(-1);
        },
    }

However it is failing with sql parser error: Expected end of statement, found: COMMENT at Line: 1, Column 691. Is there a way to ignore anything outside the parenthesis as that is irrelevant for my usecase however I suspect this is unintended behaviour?

CREATE TABLE `randoms_2` (
  `first_name` varchar(64) NOT NULL,
  `last_name` varchar(64) NOT NULL,
  `home_phone` varchar(12) NOT NULL,
  `email` varchar(64) NOT NULL,
  `address` varchar(64) NOT NULL,
  `city` varchar(32) NOT NULL,
  `state` varchar(2) NOT NULL,
  `zip` varchar(5) NOT NULL,
  `date_of_birth` varchar(10) NOT NULL,
  `ssn` varchar(9) NOT NULL,
  `drivers_license` varchar(32) NOT NULL,
  `drivers_license_state` varchar(2) NOT NULL,
  `income_type` varchar(16) NOT NULL,
  `account_type` varchar(16) NOT NULL,
  `routing_number` varchar(32) NOT NULL,
  `account_number` varchar(32) NOT NULL,
  `bank_name` varchar(64) NOT NULL,
  PRIMARY KEY (`ssn`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='LOAD DATA LOCAL INFILE "random.csv" INTO TABLE randoms_2 COLUMNS TERMINATED BY '','' ENCLOSED BY ''"'' LINES TERMINATED BY ''\\r\\n'' IGNORE 1 LINES;';
alamb commented 1 year ago

It looks to me like the parser doesn't quite handle parsing the entire set of mysql CREATE TABLE syntax. A fix would be most appreciated.

Thank you for the report @gadgetlabs

SimonMarechal commented 1 year ago

Apparently it works if I move the "comment" part of the parse_create_table function at the end. SQL options can be provided in any order, so I don't think the logic is sound. I think there should be loop where the keyword is retrieved, and then a big match statement should be used. Does that make sense?

SimonMarechal commented 1 year ago

Apparently something that looks like parse_optional_columns_option would do the trick.