xnuinside / simple-ddl-parser

Simple DDL Parser to parse SQL (HQL, TSQL, AWS Redshift, BigQuery, Snowflake and other dialects) ddl files to json/python dict with full information about columns: types, defaults, primary keys, etc. & table properties, types, domains, etc.
MIT License
179 stars 40 forks source link

Snowflake Table DDL support of WITH MASKING POLICY column definition #201

Closed dmaresma closed 1 year ago

dmaresma commented 1 year ago

Is your feature request related to a problem? Please describe. I would like to improve the CREATE TABLE statement of Snowflake ddl to support the Column WITH MASKING POLICY.

Describe the solution you'd like CREATE OR REPLACE TABLE schemaName.TableAddresses ( AddressID NUMBER(38,0) NOT NULL, Address1 VARCHAR(100) WITH MASKING POLICY DBName.MASKING_POLICY_LIBRARY.MASK_STRING WITH TAG (DBName.MASKING_POLICY_LIBRARY.PROJECT_POLICY_MASK='mask_object') ); Supported by DDLParser.run() functions

Describe alternatives you've considered add MASKING POLICY and TAG members same as Comment, Nullable or Default properties

https://docs.snowflake.com/en/sql-reference/sql/create-table

dmaresma commented 1 year ago

here the test : ` ddl = """ create TABLE ASIN.EXCLUSION ( USER_COMMENT VARCHAR(100), PROCESS_SQN NUMBER(10,0) NOT NULL, ADDRESS1 VARCHAR(100) WITH MASKING POLICY DBName.MASKING_POLICY_LIBRARY.MASK_STRING WITH TAG (DBName.MASKING_POLICY_LIBRARY.PROJECT_POLICY_MASK='mask_object') constraint PK_EXCLUSION primary key (ASIN) ) COMMENT ='ASINs to be excluded from the ASIN List File' ; """ result = DDLParser(ddl, normalize_names=True).run(output_mode="snowflake")

expected = [
    {
        "alter": {},
        "checks": [],
        "clone": None,
        "columns": [
            {
                "check": None,
                "default": None,
                "name": "USER_COMMENT",
                "nullable": True,
                "references": None,
                "size": 100,
                "type": "VARCHAR",
                "unique": False,
            },
            {
                "check": None,
                "default": None,
                "name": "PROCESS_SQN",
                "nullable": False,
                "references": None,
                "size": (10, 0),
                "type": "NUMBER",
                "unique": False,
            },
            {
                "check": None,
                "default": None,
                "name": "ADDRESS1",
                "nullable": False,
                "references": None,
                "size": 100,
                "type": "VARCHAR",
                "unique": False,
                "with_mask_policy": "DBName.MASKING_POLICY_LIBRARY.MASK_STRING",
                "with_tag" : "(DBName.MASKING_POLICY_LIBRARY.PROJECT_POLICY_MASK='mask_object')"
            },
        ],
        "constraints": {
            "primary_keys": [
                {"columns": ["ASIN"], "constraint_name": "PK_EXCLUSION"}
            ]
        },
        "comment": "'ASINs to be excluded from the ASIN List File'",
        "index": [],
        "partitioned_by": [],
        "primary_key": ["ASIN"],
        "primary_key_enforced": None,
        "schema": "ASIN",
        "table_name": "EXCLUSION",
        "tablespace": None,
    }
]
assert result == expected`
xnuinside commented 1 year ago

changes was released in version 0.31.0, thanks for opening the PR & the issue!