edasque / DynamoDBtoCSV

Dump DynamoDB data into a CSV file
Apache License 2.0
471 stars 152 forks source link

Fixes the issue where large data sets cause a heap error. #44

Closed trevor-scott closed 6 years ago

trevor-scott commented 6 years ago

Uses the first stream of data to set the headers and then just writes each unparsed batch after that. If there are less than 1000 items returned then we can be sure that the headers are all set properly. However, if the table has more than 1000 items we are assuming that they are all accounted for in the first 1000 returned.

edasque commented 6 years ago

👍