[Bug] [SQLServer CDC] Table names with underscores report an error when creating a job

Stapxs commented 10 months ago

Search before asking

[X] I searched in the issues and found nothing similar.

Flink version

Version: 1.17.2, Commit ID: c0027e5

Flink CDC version

flink-sql-connector-sqlserver-cdc-3.0.0

Database and its version

Microsoft SQL Server 2012 (SP1) - 11.0.3000.0 (Intel X86) Oct 19 2012 13:43:21 Copyright (c) Microsoft Corporation Enterprise Evaluation Edition on Windows NT 6.1 (Build 7601: Service Pack 1) (WOW64) (Hypervisor)

Minimal reproduce step

PS: Im not good at english.

Use doris-flink-connector to create jobs，and including a table with an underscore:

bin/flink run \
     -Dexecution.checkpointing.interval=60s \
     -Dparallelism.default=1 \
     -c org.apache.doris.flink.tools.cdc.CdcTools \
     ./lib/flink-doris-connector-1.17-1.5.0.jar \
     sqlserver-sync-database \
     --database db_name \
     --sqlserver-conf hostname=2.18.xxx.xxx \
     --sqlserver-conf port=1433 \
     --sqlserver-conf username=sa \
     --sqlserver-conf password="xxx" \
     --sqlserver-conf database-name=db_name \
     --sqlserver-conf schema-name=dbo \
     --including-tables "TABLE_NAME" \
     --sink-conf fenodes=fe:8030 \
     --sink-conf username=root \
     --sink-conf password=xxx \
     --sink-conf jdbc-url=jdbc:mysql://fe:9030 \
     --sink-conf doris.batch.size=512 \
     --sink-conf doris.exec.mem.limit=100000000 \
     --table-conf replication_num=1

Next, try to create another table task without underscores. There is no such problem.

What did you expect to see?

Create jon success

What did you see instead?

Got Errors:

Jan 15, 2024 12:00:08 PM com.microsoft.sqlserver.jdbc.TDSChannel enableSSL
WARNING: TLSv1 was negotiated. Please update server and client to use TLSv1.2 at minimum.

------------------------------------------------------------
 The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: null
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:105)
        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:851)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:245)
        at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1095)
        at org.apache.flink.client.cli.CliFrontend.lambda$mainInternal$9(CliFrontend.java:1189)
        at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
        at org.apache.flink.client.cli.CliFrontend.mainInternal(CliFrontend.java:1189)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1157)
Caused by: java.lang.NullPointerException
        at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:61)
        at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:37)
        at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:26)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.processExecuteResults(SQLServerStatement.java:1317)
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.processResponse(SQLServerPreparedStatement.java:543)
        at com.microsoft.sqlserver.jdbc.TDSCommand.close(IOBuffer.java:7487)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.discardLastExecutionResults(SQLServerStatement.java:147)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.closeInternal(SQLServerStatement.java:676)
        at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closeInternal(SQLServerPreparedStatement.java:337)
        at com.microsoft.sqlserver.jdbc.SQLServerStatement.close(SQLServerStatement.java:691)
        at com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getColumns(SQLServerDatabaseMetaData.java:659)
        at org.apache.doris.flink.tools.cdc.SourceSchema.<init>(SourceSchema.java:51)
        at org.apache.doris.flink.tools.cdc.sqlserver.SqlServerSchema.<init>(SqlServerSchema.java:26)
        at org.apache.doris.flink.tools.cdc.sqlserver.SqlServerDatabaseSync.getSchemaList(SqlServerDatabaseSync.java:102)
        at org.apache.doris.flink.tools.cdc.DatabaseSync.build(DatabaseSync.java:111)
        at org.apache.doris.flink.tools.cdc.CdcTools.syncDatabase(CdcTools.java:134)
        at org.apache.doris.flink.tools.cdc.CdcTools.createSqlServerSyncDatabase(CdcTools.java:96)
        at org.apache.doris.flink.tools.cdc.CdcTools.main(CdcTools.java:59)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
        ... 9 more

Anything else?

I don’t know whether this problem comes from flink, doris-flink-connector or flink-sql-connector-sqlserver-cdc. I will submit the issue here for the time being. Sorry if it bothers you.

Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

vinlee19 commented 8 months ago

Maybe, you can ask the doris community for help. Can you provide the table schema?

Stapxs commented 8 months ago

Ah. Sorry, because it has been a long time, I didn't keep the previous test environment and related data, and I can't reproduce the situation again for the time being.Thank you very much for your reply 😊

apache / flink-cdc