search

getindata / dbt-flink-adapter

Adapter for dbt that executes dbt pipelines on Apache Flink
Apache License 2.0
83 stars 10 forks source link

Add column name quotes using back tick (`) #19

Open gliter opened 1 year ago

gliter commented 1 year ago

Without backticks quotation Flink will try to interprete column names as key words, example given when column name is "timestamp" it will fail.

Flink SQL> create table clickstream (timestamp DECIMAL,user_id DECIMAL,balance DECIMAL,load_balance DECIMAL,event STRING) with (
>        'connector' = 'kafka',
>        'properties.bootstrap.servers' = 'kafka:29092',
>        'topic' = 'clickstream',
>        'value.format' = 'json',
>        'properties.group.id' = 'dbt-seed',
>        'value.json.encode.decimal-as-plain-number' = 'true'
>       
>     )
> 
> ;
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.sql.parser.impl.ParseException: Encountered "timestamp" at line 1, column 27.
Was expecting one of:
    "CONSTRAINT" ...
    "PRIMARY" ...
    "UNIQUE" ...
    "WATERMARK" ...
    <BRACKET_QUOTED_IDENTIFIER> ...
    <QUOTED_IDENTIFIER> ...
    <BACK_QUOTED_IDENTIFIER> ...
    <HYPHENATED_IDENTIFIER> ...
    <IDENTIFIER> ...
    <UNICODE_QUOTED_IDENTIFIER> ...

Flink SQL> create table clickstream (`timestamp` DECIMAL,user_id DECIMAL,balance DECIMAL,load_balance DECIMAL,event STRING) with (
>        'connector' = 'kafka',
>        'properties.bootstrap.servers' = 'kafka:29092',
>        'topic' = 'clickstream',
>        'value.format' = 'json',
>        'properties.group.id' = 'dbt-seed',
>        'value.json.encode.decimal-as-plain-number' = 'true'
>       
>     )
> ;
[INFO] Execute statement succeed.
gliter commented 1 year ago

I see that we have sometimes quotes inside macros and sometimes are taken from function that uses QuotePolicy. We should make approach uniform across all places.