Closed shirley-gu-timeplus closed 1 month ago
I know what happened, the table column naming conflict. minimal reproduce case:
create stream ge(id int, Id int)
create stream ge2(id int, Id int)
insert into ge(id, Id) values(1,2)
./sling run --src-conn PROTON --src-stream 'ge' --tgt-conn PROTON --tgt-object "ge2" --mode incremental --update-key _tp_time -d
log:
2024-10-23 08:13:45 WRN Operation failed, retrying in 709.425969ms: ~ could not insert into `default`.`ge2`.
--- task_run.go:112 func1 ---
--- task_run.go:562 runDbToDb ---
--- task_run.go:822 runProtonToProton ---
--- task_run.go:845 retryWithBackoff ---
--- retry.go:49 RetryNotify ---
--- retry.go:61 RetryNotifyWithTimer ---
--- retry.go:88 ] ---
--- retry.go:18 func1 ---
--- task_run.go:823 func5 ---
--- task_run.go:440 runFileToDB ---
--- task_run_write.go:146 WriteToDb ---
--- task_run_write.go:554 writeDirectly ---
--- database.go:2313 BulkImportFlow ---
--- database.go:2300 func1 ---
--- database_proton.go:233 BulkImportStream ---
~ could not bulk import
--- database.go:1556 GetColumns ---
~ could not get matching list of columns from table
--- database.go:1496 GetTableColumns ---
provided field 'id1' not found in table `default`.`ge2`
context canceled
^C
Issue Description
Description of the issue: When import data into target table, unexpected column name found which cause import failure issue1: No such column a1 in stream default.gj_rtdc_oms_order_statistics issue2: provided field 'securityid1' not found in table
default
.gj_i_futurePositionAmount_fix3
Sling version (
sling --version
): 1.2.20-rc10-timeplus 1.2.20-rc8-timeplusOperating System (
linux
,mac
,windows
): linux