Open AshinGau opened 9 months ago
We should not use Hive catalog, that's why we introduce a HoodieHiveCatalog
where we do many tasks for createTable
.
We should not use Hive catalog, that's why we introduce a
HoodieHiveCatalog
where we do many tasks forcreateTable
.
OK, I would strongly recommend that users use the hudi catalog and no longer maintain abnormal behavior in the hive catalog. How about the unexpected result of querying _ro
table created by hudi catalog? Is it a bug in flink-sql?
Probably, can you show the table paramerers read from Hudi hive catalog for the problematic ro table?
@AshinGau An updates here?
Describe the problem you faced
SHOW CREATE TABLE
._ro
table is the same as_rt
table, but spark return the different results when querying_ro
table.To Reproduce Flink 1.17.2 + Hudi 0.14.1
Steps to reproduce the behavior:
-- hudi catlaog create catalog hudi with ( 'type'='hudi', 'catalog.path' = 'hdfs://xxx/hudi_flink_hive_catalog', 'hive.conf.dir' = '/usr/local/service/hive/conf', 'mode'='hms');
use catalog hive; use hudi_flink; CREATE TABLE hive_ctl_table( ts BIGINT, uuid VARCHAR(40) PRIMARY KEY NOT ENFORCED, rider VARCHAR(20), driver VARCHAR(20), fare DOUBLE, city VARCHAR(20) ) -- PARTITIONED BY (
city
) // flink 1.17 can insert the partitioned table, but throws errors when querying. It works will in flink 1.14 WITH ( 'connector' = 'hudi', 'path' = 'hdfs://xxx/hudi_flink.db/hive_ctl_table', 'table.type' = 'MERGE_ON_READ' );use catalog hudi; use hudi_flink; CREATE TABLE hudi_ctl_table( ts BIGINT, uuid VARCHAR(40) PRIMARY KEY NOT ENFORCED, rider VARCHAR(20), driver VARCHAR(20), fare DOUBLE, city VARCHAR(20) ) PARTITIONED BY (
city
) WITH ( 'connector' = 'hudi', 'path' = 'hdfs://xxx/hudi_ctl_table', 'table.type' = 'MERGE_ON_READ' );