timeplus-io / sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
https://docs.slingdata.io
GNU General Public License v3.0
1 stars 0 forks source link

column name issue when import data to target table #34

Closed shirley-gu-timeplus closed 1 month ago

shirley-gu-timeplus commented 1 month ago

Issue Description

yokofly commented 1 month ago

I know what happened, the table column naming conflict. minimal reproduce case:

create stream ge(id int, Id int)
create stream ge2(id int, Id int)
insert into ge(id, Id) values(1,2)
 ./sling run --src-conn PROTON  --src-stream 'ge' --tgt-conn PROTON  --tgt-object "ge2" --mode incremental --update-key _tp_time  -d

log:

2024-10-23 08:13:45 WRN Operation failed, retrying in 709.425969ms: ~ could not insert into `default`.`ge2`.
--- task_run.go:112 func1 ---
--- task_run.go:562 runDbToDb ---
--- task_run.go:822 runProtonToProton ---
--- task_run.go:845 retryWithBackoff ---
--- retry.go:49 RetryNotify ---
--- retry.go:61 RetryNotifyWithTimer ---
--- retry.go:88 ] ---
--- retry.go:18 func1 ---
--- task_run.go:823 func5 ---
--- task_run.go:440 runFileToDB ---
--- task_run_write.go:146 WriteToDb ---
--- task_run_write.go:554 writeDirectly ---
--- database.go:2313 BulkImportFlow ---
--- database.go:2300 func1 ---
--- database_proton.go:233 BulkImportStream ---
~ could not bulk import
--- database.go:1556 GetColumns ---
~ could not get matching list of columns from table
--- database.go:1496 GetTableColumns ---
provided field 'id1' not found in table `default`.`ge2`

context canceled
^C