singer-io / target-csv

Write Singer data to CSV files

GNU Affero General Public License v3.0

37 stars 68 forks source link

Rewrite files with correct header at end of stream #27

Closed abij closed 3 weeks ago

abij commented 4 years ago

Description of change

When the fields of the flattened record are containing new fields, they are added in the csv-file header. (Rewriting the csv). The header for the stream is updated according to the new header in the csv-file.

Links to Issue https://github.com/singer-io/target-csv/issues/3

Manual QA steps

Risks

Performance: Large CSV's will suffer from rewrite if records have new fields. (rewrite happens at and of stream)

Rollback steps

revert this branch