MeltanoLabs / target-snowflake

Singer Target for the Snowflake cloud Data Warehouse
https://hub.meltano.com/loaders/target-snowflake--meltanolabs/
Other
10 stars 24 forks source link

cannot change column from type NUMBER to VARCHAR #276

Open sebastianswms opened 1 month ago

sebastianswms commented 1 month ago

Running on Arch, tap-distributor-srs target-snowflake, error:

{"consumer": true, "producer": false, "string_id": "target-snowflake", "cmd_type": "elb", "run_id": "2c975dcb-7ea3-48f4-8db1-e292c2b0511d", "job_name": "dev:tap-distributor-srs-to-target-snowflake:pipeline-ZGxyhvN8", "stdio": "stderr", "name": "target-snowflake", "event": "snowflake.connector.errors.ProgrammingError: 002108 (22000): 01b7d359-0204-e3d8-0001-6b5a013e0b62: SQL compilation error: cannot change column _SDC_ORDER_NUMBER from type NUMBER(38,0) to VARCHAR(16777216)", "level": "info", "timestamp": "2024-10-20T22:49:51.964977Z"}
[...]
{"consumer": true, "producer": false, "string_id": "target-snowflake", "cmd_type": "elb", "run_id": "2c975dcb-7ea3-48f4-8db1-e292c2b0511d", "job_name": "dev:tap-distributor-srs-to-target-snowflake:pipeline-ZGxyhvN8", "stdio": "stderr", "name": "target-snowflake", "event": "[SQL: ALTER TABLE \"ARCH_RAW\".\"[distributor name goes here]\".\"ORDER\" ALTER COLUMN _sdc_order_number SET DATA TYPE VARCHAR]", "level": "info", "timestamp": "2024-10-20T22:49:51.978192Z"}

Coercion from NUMBER to VARCHAR should be a fairly simple process, I would think. Not sure what's going on and I didn't dive in.

edgarrmondragon commented 1 month ago

By looking at the error message, it seems like a limitation on Snowflake itself.

visch commented 1 month ago

image Looks like you're right edgar, probably have to do a different series of sql commands in snowflake to make this happen