Unimplemented error from map in array of structs

[dougscher]

The following hive schema throws an unimplemented exception: ALTER TABLE TABLE CHANGE ctx ctx struct< action:struct<fld1:string, arr1:array<struct<fld2:string, fld3:string, map1:map<string, string>>>>

If you remove map1, it works. Not sure if this is a serde issue or a hive issue.

[dougscher]

Can you please look into this. The workaround is nasty. It looks like there is a failure in handling a map in an array in a struct. Thanks, Doug

[rcongiu]

So, for starters, it is not a SerDe issue, the SerDe can handle nested Maps just fine, it's Hive actually that does not allow to change changing the type of a column in an incompatible way (see https://issues.apache.org/jira/browse/HIVE-4409). Second, please next time you submit an issue, please try to supply as much info as possible to reproduce the issue, including:

• CREATE TABLE statement
• data sample
• stack trace of the error you're getting.

[dougscher]

This failure was occurring on empty tables that had just been populated. I am aware that old data can trip up an altered schema...

Can you please verify that this json is compatible with the serde: { action: { fld1: 'fld1', arr1: [{ fld2: 'fld2', fld3: 'fld3', map1: { m1: 'm1', m2: 'm2' } }] } }

Assuming a schema of: struct<action:struct<fld1:string, arr1:array<struct<fld2:string, fld3:string, map1:map<string, string>>>>

Thanks, Doug

[rcongiu]

It doesn't matter if it does have data or not, hive won't allow an incompatible schema change. Data is compatible with the serde. Create the table with the right schema at the beginning and load the data into it , you'll see it works.