transferwise / pipelinewise-target-redshift

Singer.io Target for Amazon Redshift - PipelineWise compatible
https://transferwise.github.io/pipelinewise/
Other
12 stars 65 forks source link

Support for flatten arrays #27

Open pcorbel opened 5 years ago

pcorbel commented 5 years ago

Describe the feature Currently, when we got an array, it is stored as a string in Redshift.

Example:

{"hello": "world", "array_of_properties": [10, 20, 30]}

will be stored as

hello array_of_properties
world [10, 20 , 30]

Some other redshift-target proposes a flatten feature where the output will be like

hello array_of_properties_id
world abcdef
source_properties_id level_id value
abcdef 0 10
abcdef 1 20
abcdef 2 30

It would be nice to have this feature via a property like does_flatten_arrays in the config.json file

gbachRM commented 3 years ago

This would be amazing if it could be implemented!!