awslabs / emr-dynamodb-connector

Implementations of open source Apache Hadoop/Hive interfaces which allow for ingesting data from Amazon DynamoDB
Apache License 2.0
217 stars 135 forks source link

Struct fields become lowercase in Hive #138

Open sgalla1 opened 4 years ago

sgalla1 commented 4 years ago

When reading a Map type from DynamoDB into a Struct type in Hive, field names become lowercase. This causes issues if then writing back to another table in Dynamo.

E.g. A reading Dynamo data:

    {
        "Frame":"Red",
        "Wheels":"Black"
    }

Into hive column with type struct<Frame:string,Wheels:string> will become:

    {
        "frame":"Red",
        "wheels":"Black"
    }

If written back to DynamoDB. This is handled for column names through the dynamodb.column.mapping but I see no way to map the struct field names back to map fields.