[Bug] [CDC-SqlServer] Not support SqlServer type 'datetimeoffset'

CheneyYin commented 1 year ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

CREATE DATABASE column_type_test;
USE column_type_test;
EXEC sys.sp_cdc_enable_db;

CREATE TABLE full_types (
    val_datetimeoffset datetimeoffset(4),
);

Not support SqlServer type datetimeoffset .

Related issues:

Not support SqlServer type timestamp: #5449
Not support SqlServer type varbinary: #5544

SeaTunnel Version

dev

SeaTunnel Config

env {
  # You can set engine configuration here
  execution.parallelism = 1
  job.mode = "STREAMING"
  execution.checkpoint.interval = 5000
}

source {
  SqlServer-CDC {
    result_table_name = "tbl"
    username = "****"
    password = "*******"
    database-names = ["column_type_test"]
    table-names = ["column_type_test.dbo.full_types"]
    base-url = "jdbc:sqlserver://localhost:1433;databaseName=column_type_test"
  }
}

transform {
}

sink {
  console {
    source_table_name = "tbl"
  }
}

Running Command

example

Error Exception

2023-09-09 13:49:27,298 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-473233] [5.1] [localhost]:5801 is SHUTDOWN
2023-09-09 13:49:27,298 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed HazelcastInstance ......
2023-09-09 13:49:27,298 ERROR org.apache.seatunnel.core.starter.SeaTunnel - 

===============================================================================

2023-09-09 13:49:27,298 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error, 

2023-09-09 13:49:27,298 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/seatunnel/issues

2023-09-09 13:49:27,298 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed 

2023-09-09 13:49:27,301 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:190)
    at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
    at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:43)
Caused by: java.lang.UnsupportedOperationException: Don't support SqlSever type 'datetimeoffset' yet, jdbcType:'-155'.
    at org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.utils.SqlServerTypeUtils.convertFromColumn(SqlServerTypeUtils.java:74)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550)
    at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
    at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)
    at org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.utils.SqlServerTypeUtils.convertFromTable(SqlServerTypeUtils.java:88)
    at org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.source.SqlServerIncrementalSource.createDebeziumDeserializationSchema(SqlServerIncrementalSource.java:125)
    at org.apache.seatunnel.connectors.cdc.base.source.IncrementalSource.prepare(IncrementalSource.java:119)
    at org.apache.seatunnel.engine.core.parse.JobConfigParser.parseSource(JobConfigParser.java:85)
    at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:317)
    at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:179)
    at org.apache.seatunnel.engine.core.job.AbstractJobEnvironment.getLogicalDag(AbstractJobEnvironment.java:109)
    at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:73)
    at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:142)
    ... 2 more

Zeta or Flink or Spark Version

zeta

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

EricJoy2048 commented 1 year ago

Yes, It's a bug.

zhilinli123 commented 11 months ago

If patch has not been repaired, would you be willing to assign the problem to me? I'm not sure anyone is fixing it right now @EricJoy2048 @hailin0 @CheneyYin @Carl-Zhou-CN

CheneyYin commented 11 months ago