4paradigm / OpenMLDB

OpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference.
https://openmldb.ai
Apache License 2.0
1.59k stars 321 forks source link

load hive data into openmldb error: requirement failed: schema mismatch #3981

Open chen131311 opened 2 months ago

chen131311 commented 2 months ago

Bug Description openMLDB:0.9.2 load hive data into openmldb error: requirement failed: schema mismatch

Expected Behavior

Relation Case

Steps to Reproduce

  1. hive table : CREATE EXTERNAL TABLE tmp.ods_admin_da( id bigint, biz_key string ', biz_domain string , biz_type string , locked string , url string , method string , created_at string, updated_at string, biz_tag string, status string)
  2. openMLDB table: desc test4;

    Field Type Null Default


    1 id BigInt YES 2 biz_key Varchar YES 3 biz_domain Varchar YES 4 biz_type Varchar YES 5 locked Varchar YES 6 url Varchar YES 7 method Varchar YES 8 created_at Varchar YES 9 updated_at Varchar YES 10 biz_tag Varchar YES 11 status Varchar YES

  3. LOAD DATA INFILE 'hive://tmp.ods_admin_da' INTO TABLE demo_db.test4 OPTIONS(deep_copy=true,mode='append');
  4. Get exception: requirement failed: schema mismatch, loaded StructType(StructField(id,LongType,true),StructField(biz_key,StringType,true),StructField(biz_domain,StringType,true),StructField(biz_type,StringType,true),StructField(locked,StringType,true),StructField(url,StringType,true),StructField(method,StringType,true),StructField(created_at,StringType,true),StructField(updated_at,StringType,true),StructField(biz_tag,StringType,true),StructField(status,StringType,true)) != table StructType(StructField(id,LongType,true),StructField(biz_key,StringType,true),StructField(biz_domain,StringType,true),StructField(biz_type,StringType,true),StructField(locked,StringType,true),StructField(url,StringType,true),StructField(method,StringType,true),StructField(created_at,StringType,true),StructField(updated_at,StringType,true),StructField(biz_tag,StringType,true),StructField(status,StringType,true)), check hive://kc_tmp.ods_admin_admin_biz_config_da
Shouren commented 1 month ago

@chen131311 I am trying to reproduce this bug and i want to know how is the table test4 created? Please provide the DDL of this table