rcongiu / Hive-JSON-Serde

Read - Write JSON SerDe for Apache Hive.
Other
733 stars 391 forks source link

Just load first element when the json is an array #239

Open wangxia514 opened 1 year ago

wangxia514 commented 1 year ago

data

[{"key1":"value1a","key2":"value2a",...},{"key1":"value1b","key2":"value2b",...}]

create external table

CREATE EXTERNAL TABLE IF NOT EXISTS testdata.trynum85 (
  data array<struct<`key1`:string,
                    `key2`:string>>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' 
WITH SERDEPROPERTIES (
  'serialization.format' = '1',
  'strip.outer.array'='false'
) LOCATION 's3://mys3/new-json-structure/'
TBLPROPERTIES ('has_encrypted_data'='false');

result: the logic only finds the first object in the array

SELECT stuff.key1
FROM "testdata"."trynum85 ", UNNEST(data) AS t(stuff)
returns
key1
----
value1a

Hi @rcongiu , would you please have a look, thanks