search

transferwise / pipelinewise-target-redshift

Singer.io Target for Amazon Redshift - PipelineWise compatible
https://transferwise.github.io/pipelinewise/
Other
12 stars 65 forks source link

Log inserts, updates and csv size_bytes #52

Closed koszti closed 4 years ago

koszti commented 4 years ago

Redshift equivalent of the Snowflake PR https://github.com/transferwise/pipelinewise-target-snowflake/pull/69

For better logging experience adding the csv file size to the logs and logging load stats in a more consumable JSON format:

time=2020-04-17 17:31:14 name=target_redshift level=INFO message=Loading into local_dev1."TEST_TABLE_THREE": {"inserts": 3, "updates": 0, "size_bytes": 57}
time=2020-04-17 17:31:14 name=target_redshift level=INFO message=Loading into local_dev1."TEST_TABLE_TWO": {"inserts": 2, "updates": 0, "size_bytes": 60}
time=2020-04-17 17:31:14 name=target_redshift level=INFO message=Loading into local_dev1."TEST_TABLE_ONE": {"inserts": 1, "updates": 0, "size_bytes": 8}