shinichi-takii / ddlparse

DDL parase and Convert to BigQuery JSON schema and DDL statements
https://pypi.org/project/ddlparse/
BSD 3-Clause "New" or "Revised" License
87 stars 29 forks source link

Unable to parse MySQL foreign key constraints (ON DELETE, CASCADE) #56

Closed byarmis closed 4 years ago

byarmis commented 4 years ago

ddlparse appears to be unable to correctly parse the following DDL:

CREATE TABLE `foo`
(
    `id`            INT(10) UNSIGNED NOT NULL,
    `some_other_id` INT(10) UNSIGNED NOT NULL,
    UNIQUE KEY `foo` (`id`, `some_other_id`),
    CONSTRAINT `fk_1` FOREIGN KEY (`id`)
        REFERENCES `bar` (`id`)
        ON DELETE CASCADE
        ON UPDATE NO ACTION
) ENGINE = InnoDB
  DEFAULT CHARSET = latin1;

When I replace the example DDL with the one above, the output is:

Traceback (most recent call last):
  File "ddl.py", line 27, in <module>
    print(table.to_bigquery_fields())
  File "/Users/byarmis/venvs/bootstrapper/lib/python3.6/site-packages/ddlparse/ddlparse.py", line 506, in to_bigquery_fields
    return self._columns.to_bigquery_fields(name_case)
  File "/Users/byarmis/venvs/bootstrapper/lib/python3.6/site-packages/ddlparse/ddlparse.py", line 444, in to_bigquery_fields
    bq_fields.append(col.to_bigquery_field(name_case))
  File "/Users/byarmis/venvs/bootstrapper/lib/python3.6/site-packages/ddlparse/ddlparse.py", line 371, in to_bigquery_field
    type = self.bigquery_legacy_data_type
  File "/Users/byarmis/venvs/bootstrapper/lib/python3.6/site-packages/ddlparse/ddlparse.py", line 335, in bigquery_legacy_data_type
    return self.bigquery_data_type
  File "/Users/byarmis/venvs/bootstrapper/lib/python3.6/site-packages/ddlparse/ddlparse.py", line 329, in bigquery_data_type
    raise ValueError("Unknown data type : '{}'".format(self._data_type))
ValueError: Unknown data type : 'DELETE'

Skipping past the BigQuery aspects (commenting out everything from # parse pattern (1-2) through to print("* COLUMN *") as well as anything mentioning BigQuery in the example body), the parsed columns are as follows:

* COLUMN *
{
  "name": "id",
  "data_type": "INT",
  "length": 10,
  "precision(=length)": 10,
  "scale": null,
  "is_unsigned": true,
  "is_zerofill": false,
  "constraint": "NOT NULL UNIQUE",
  "not_null": true,
  "PK": false,
  "unique": true,
  "auto_increment": false,
  "distkey": false,
  "sortkey": false,
  "encode": null,
  "default": null,
  "comment": null,
  "description(=comment)": null
}
{
  "name": "some_other_id",
  "data_type": "INT",
  "length": 10,
  "precision(=length)": 10,
  "scale": null,
  "is_unsigned": true,
  "is_zerofill": false,
  "constraint": "NOT NULL UNIQUE",
  "not_null": true,
  "PK": false,
  "unique": true,
  "auto_increment": false,
  "distkey": false,
  "sortkey": false,
  "encode": null,
  "default": null,
  "comment": null,
  "description(=comment)": null
}
{
  "name": "ON",
  "data_type": "DELETE",
  "length": null,
  "precision(=length)": null,
  "scale": null,
  "is_unsigned": false,
  "is_zerofill": false,
  "constraint": "",
  "not_null": false,
  "PK": false,
  "unique": false,
  "auto_increment": false,
  "distkey": false,
  "sortkey": false,
  "encode": null,
  "default": null,
  "comment": null,
  "description(=comment)": null
}
{
  "name": "CASCADE",
  "data_type": "ON",
  "length": null,
  "precision(=length)": null,
  "scale": null,
  "is_unsigned": false,
  "is_zerofill": false,
  "constraint": "",
  "not_null": false,
  "PK": false,
  "unique": false,
  "auto_increment": false,
  "distkey": false,
  "sortkey": false,
  "encode": null,
  "default": null,
  "comment": null,
  "description(=comment)": null
}
{
  "name": "UPDATE",
  "data_type": "NO",
  "length": null,
  "precision(=length)": null,
  "scale": null,
  "is_unsigned": false,
  "is_zerofill": false,
  "constraint": "",
  "not_null": false,
  "PK": false,
  "unique": false,
  "auto_increment": false,
  "distkey": false,
  "sortkey": false,
  "encode": null,
  "default": null,
  "comment": null,
  "description(=comment)": null
}

This is a valid MySQL DDL in both 5.7 (what I'm using) as well as 8.0 (the current version). I am running ddlparse v1.7.0

shinichi-takii commented 4 years ago

@byarmis Fixed this issue in v1.8.0