Handle complex JSON objects

[GoogleCodeExporter]

Deserializing basic JSON objects with simple key/values is fine, but it
cannot handle nested objects or arrays. Hive can support complex objects so
this SerDe should too.

Original issue reported on code.google.com by pas...@gmail.com on 16 Feb 2010 at 9:58

[GoogleCodeExporter]

Are there any plans to fix this?

Thanks
Parag

Original comment by parag.ar...@gmail.com on 28 Jun 2010 at 7:17

[GoogleCodeExporter]

Hi Parag,

It turns out that the code that has been written already supports map objects 
within a JSON object. When I wrote this issue, I just assumed it didn't, but 
since then, we have tested it and it works.

I need to include the tests for it though.

Original comment by pas...@gmail.com on 28 Jun 2010 at 9:25

[GoogleCodeExporter]

Hey

Probably I am missing something. 

According to me while writing my own tests, nested queries are not supported by 
org.Json.JsonObject.get(String  key)

Changing the function to 

public Object get(String key) throws JSONException {
        Object o = opt(key);
        if (o == null) {
            if(key.contains("/"))
            {
                StringTokenizer st = new StringTokenizer(key, "/");
                String parentKey = st.nextToken();
                o = get(parentKey);
                JSONObject obj = new JSONObject(new JSONTokener(o.toString()));
                o = obj.get(st.nextToken());
            }
            else
            {
                throw new JSONException("JSONObject[" + quote(key) + "] not found.");
            }
        }
        return o;
    }

solves the problem though. 

Nested json value for key3 from mentioned json

 {"key1":{"key2":{"key3":"value}}} can be obtained by giving key as key1/key2/key3. 

Thanks so much for sharing the code. Hive serde now supports nested complex 
json to n level :)

Original comment by parag.ar...@gmail.com on 28 Jun 2010 at 10:12

[GoogleCodeExporter]

I am still having trouble dealing with arrays and maps.

Here's my session transcript:

hive> add jar ./hive-json-serde.jar;
Added ./hive-json-serde.jar to class path
hive> create table test_table (key int, c1 array <string>, c2 map <string, 
string>) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde';
Found class for org.apache.hadoop.hive.contrib.serde2.JsonSerde
OK
Time taken: 5.297 seconds
hive> load data local inpath './test.json' overwrite into table test_table;
Copying data from file:/home/riyer/hive/test.json
Loading data to table test_table
OK
Time taken: 0.532 seconds
hive> select * from test_table;
OK
Failed with exception java.io.IOException:java.lang.ClassCastException: 
org.json.JSONArray cannot be cast to java.util.List
Time taken: 0.179 seconds

Contents of test.json:

{ "key":123,"c1":["s1","s2","s3"],"c2":{"k1":"v1","k2":"v2"}}
{ "key":456,"c1":["s4","s5","s6"],"c2":{"k3":"v3","k4":"v4"}}

Original comment by bha...@gmail.com on 13 Aug 2010 at 7:33

[GoogleCodeExporter]

any update wrt comment #4? I am getting the same error..
Failed with exception java.io.IOException:java.lang.ClassCastException: 
org.json.JSONArray cannot be cast to java.util.List

Original comment by anurag.p...@gmail.com on 28 Oct 2010 at 10:55

[GoogleCodeExporter]

I can't get the nested json to work. 
Can you please give an example of the CREATE EXTERNAL TABLE 
and a SELECT

Original comment by ayonsi...@gmail.com on 29 Jul 2011 at 6:17

[GoogleCodeExporter]

We just ran into this same problem with an array.  Any hope in getting it fixed?

Original comment by mcor...@gmail.com on 9 Sep 2011 at 8:21

[GoogleCodeExporter]

Just wondering, if complex json objects (nested) ever implemented. Thanks

Original comment by AmitRai2...@gmail.com on 6 May 2013 at 10:55

[GoogleCodeExporter]

I found that if I use the trunk and make a build the arrays work, but if I use 
the version in the downloads it didn't work.

Original comment by thriceth...@gmail.com on 7 Aug 2013 at 7:22