migrate to meltanolabs target snowflake

meltano / squared

Where the Meltano team runs Meltano! Get it???

25 stars 6 forks source link

migrate to meltanolabs target snowflake #647

Closed pnadolny13 closed 1 year ago

pnadolny13 commented 1 year ago

Theres still a few bugs that need to get resolved prior to this merging and this PR is pinned to a non-main branch so we'll have to merge those changes too.

Known Issues:

[x] dynamodb will fail due to https://github.com/MeltanoLabs/target-snowflake/issues/54
[x] spreadsheets anywhere was having issues with the mapper ID column not being present in records, it might have been related to my testing setup though.
[x] https://github.com/MeltanoLabs/target-snowflake/issues/29

pnadolny13 commented 1 year ago

@tayloramurphy This passes when theres no tables existing but when I did a test against clones of my prod tables I'm seeing some different errors when trying to alter or add columns that will immediately block this from working https://github.com/MeltanoLabs/target-snowflake/issues/59.

pnadolny13 commented 1 year ago

Pending https://github.com/MeltanoLabs/target-snowflake/pull/63

Will need to manually alter slack-public threads table because its a float and we want to make it a string but altering wont work, I'll have to cast it manually. Alter column name, create new column with original name, update all rows in new column as old value cast to string, drop old altered column.