search

rcongiu / Hive-JSON-Serde

Read - Write JSON SerDe for Apache Hive.
Other
733 stars 391 forks source link

Top level array #215

Closed YevheniiStepaniuk closed 5 years ago

YevheniiStepaniuk commented 5 years ago

This serde doesn't work with top-level arrays:

[ { "a": 32}, {"a": 31"}]

I've faced with this issue while using your serde, and no way to convert JSON files to valid, through big size.

How it can be resolved?

rcongiu commented 5 years ago

It works as long as there's one object per line. This is a limitation from hadoop, since otherwise files would not be splittable.

"Good judgment comes from experience. Experience comes from bad judgment"

On Monday, April 29, 2019, 8:49:05 AM PDT, Yevhenii Stepaniuk <notifications@github.com> wrote:

This serde doesn't work with top-level arrays: [ { "a": 32}, {"a": 31"}] I've faced with this issue while using your serde, and no way to convert JSON files to valid, through big size.

Have can it be resolved?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

YevheniiStepaniuk commented 5 years ago

@rcongiu Thanks for the answer, did not know about Hadoop restriction for JSON.