slingdata-io / sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
https://docs.slingdata.io
GNU General Public License v3.0
398 stars 27 forks source link

Strange behavior in 1.1.14 with id column drop and rename #236

Closed Daniel-Vetter-Coverwhale closed 6 months ago

Daniel-Vetter-Coverwhale commented 6 months ago

Saw a weird thing happen today. When using sling replication for a table from our mysql database with custom sql to a sql server database, we wound up getting the following error. The custom sql isn't anything fancy, just some type shifting and column truncating to fit within sql server and BCPs special needs.

Two really strange things

It could be something with the table itself, but because it worked on an earlier version I figured I'd raise an issue here and see what thoughts folks had.

Thanks a ton for sling, it's been awesome (and fast), especially with Dagster's embedded ELT.

fatal:
~ failure running replication (see docs @ https://docs.slingdata.io/sling-cli)

--------------------------- transportation_submissions ---------------------------
~ Could not execute SQL

~ Error executing: alter table "prod_db_replication"."table_name" drop column "id";
mssql: ALTER TABLE DROP COLUMN id failed because one or more objects access this column.

~ Error executing: EXEC sp_rename 'prod_db_replication.table_name.id_0Jr', 'id', 'COLUMN'
mssql: Either the parameter @objname is ambiguous or the claimed @objtype (COLUMN) is wrong.
flarco commented 6 months ago

Thanks for the details and the kind words. The target.options.adjust_column_type was switched to being true by default in 1.2.1. That's why you're seeing this. This was a mistake to set it true by default, and will revert to being false by default in the imminent 1.1.15. So, if you specify false for target.options.adjust_column_type, it should work just fine.

flarco commented 6 months ago

Closing, feel free to re-open.