transferwise / pipelinewise-target-redshift

Singer.io Target for Amazon Redshift - PipelineWise compatible
https://transferwise.github.io/pipelinewise/
Other
12 stars 65 forks source link

Support for column names with reserved keywords #18

Open mlavoie-sm360 opened 5 years ago

mlavoie-sm360 commented 5 years ago

Describe the bug

Syncing tables that have column names that are identical to reserved keywords brakes synchronization.

Steps To Reproduce

Using a test table defined such as this one as a source, would cause an error on import to redshift.

CREATE TABLE test (
    id int auto_increment
        primary key,
    new tinyint(1) default '0' not null,
    demo tinyint(1) default '0' not null
)
charset=utf8
;

Expected behavior

We would expect the table to load correctly regardless of the reserved keyword found in the column name

Screenshots and log output

-------------------------------------------------------
SYNC FINISHED - SUMMARY
-------------------------------------------------------
Total tables selected to sync  : 2
Tables loaded successfully     : 1
Exceptions during table sync   : ['schema.table: syntax error at or near "new"\nLINE 1: ...OLEAN, en BOOLEAN, demo CHARACTER VARYING(10000), new CHARAC...\n                                                             ^\n']
CPU cores                      : 4
Runtime                        : 0:00:06.355967
-------------------------------------------------------

Possible solution

Always wrap column names using double quotes, ie: "new" on CREATE statement