apache / doris-flink-connector

Flink Connector for Apache Doris
https://doris.apache.org/
Apache License 2.0
315 stars 221 forks source link

[Bug] Doris include table pattern null #294

Open gilboom opened 8 months ago

gilboom commented 8 months ago

Search before asking

Version

Doris flink connector: flink-doris-connector-1.17-1.5.0

What's Wrong?

When I use --single-sink flag to start my flink job, the task will report error:

Caused by: java.lang.IllegalArgumentException: Can't find any matched tables, please check your configured database-name: [svc_dw] and table-name: [((svc_dw)\.(null))(?!(svc_dw\.(upload_histories|supported_schemas))$)]

I try to figure out the question and I found the code that causes the question:

    protected String getSyncTableList(List<String> syncTables) {
        if (!singleSink) {
            return syncTables.stream()
                    .map(v -> getTableListPrefix() + "\\." + v)
                    .collect(Collectors.joining("|"));
        }

        // includingTablePattern and ^excludingPattern
        String includingPattern =
                String.format("(%s)\\.(%s)", getTableListPrefix(), includingTables);
        if (StringUtils.isNullOrWhitespaceOnly(excludingTables)) {
            return includingPattern;
        } else {
            String excludingPattern =
                    String.format("?!(%s\\.(%s))$", getTableListPrefix(), excludingTables);
            return String.format("(%s)(%s)", includingPattern, excludingPattern);
        }
    }

If I use --excluding-tables flag but not use --including-tables flag with --single-sink flag,the mysql flink cdc source will get the --table-name with value: ((svc_dw)\.(null))(?!(svc_dw\.(upload_histories|supported_schemas))$)

What You Expected?

Start the flink job successfully with --single-sink and --excluding-tables flags.

How to Reproduce?

Start a flink job with --single-sink and --excluding-tables flags and without --including-tables flag.

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

JNSimba commented 5 months ago

Can you try the latest version 1.6.0?