search

StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
8.91k stars 1.79k forks source link

flink-starrocks-connector 有计划支持 catalog么 #28180

Closed zouchangzhen closed 8 months ago

zouchangzhen commented 1 year ago

General Question

Dshadowzh commented 1 year ago

Yes. we have a pr https://github.com/StarRocks/starrocks-connector-for-apache-flink/pull/175, still working on to merge this.

zouchangzhen commented 1 year ago

Yes. we have a pr StarRocks/starrocks-connector-for-apache-flink#175, still working on to merge this.

这样创建catalog语句么？ CREATE CATALOG doris_catalog WITH( 'type' = 'starrocks-catalog', 'jdbc-url' = 'jdbc:mysql://:9030,:9030,*:9030', 'default-database' = '@', 'database-name' = '@', 'username' = '@', 'table-name' = '@', 'password' = '@' ); 为什么还要table-name呢

zouchangzhen commented 1 year ago

catalog 好像不支持主键模型？ insert into sink_sr.sdp_test.sink_test_zj/+ OPTIONS( 'sink.parallelism' = '3','sink.properties.partial_update' = 'true','sink.properties.columns' = 'f_sequence,f_random,f_random_str,__op') /(f_sequence,f_random,f_random_str,ts) select f_sequence,f_random,concat('testcatalog20230808sdp',f_random_str),ts from source_kafka <<< java.lang.IllegalArgumentException: Primary keys not defined in the sink TableSchema. at com.starrocks.connector.flink.manager.StarRocksSinkTable.validateTableStructure(StarRocksSinkTable.java:83) at com.starrocks.connector.flink.table.sink.StarRocksDynamicSinkFunctionV2.(StarRocksDynamicSinkFunctionV2.java:70) at com.starrocks.connector.flink.table.sink.SinkFunctionFactory.createSinkFunction(SinkFunctionFactory.java:156) at com.starrocks.connector.flink.table.sink.StarRocksDynamicTableSink.getSinkRuntimeProvider(StarRocksDynamicTableSink.java:44) at org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecSink.createSinkTransformation(CommonExecSink.java:149) at org.apache.flink.table.planner.plan.nodes.exec.stream.StreamExecSink.translateToPlanInternal(StreamExecSink.java:176) at org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:158) at

github-actions[bot] commented 8 months ago

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!