MeltanoLabs / target-athena

Singer.io Target for AWS Athena.
Other
5 stars 16 forks source link

Add support for json/jsonl format #7

Closed andrewcstewart closed 3 years ago

andrewcstewart commented 3 years ago

Athena also supports JSON/JSONL as a file format. This option may address issue #6 (CSV column shift issue).

See Athena documentation: https://docs.aws.amazon.com/athena/latest/ug/parsing-JSON.html

Adding support for JSON format would involve the following:

Example of JSON table definition:

CREATE EXTERNAL TABLE json_table (
  column_a string,
  column_b int
 )
 ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
 WITH SERDEPROPERTIES ('ignore.malformed.json' = 'true')
 LOCATION 's3://bucket/path/';