edasque / DynamoDBtoCSV

Dump DynamoDB data into a CSV file
Apache License 2.0
470 stars 152 forks source link

Dummy line #31

Closed ghost closed 6 years ago

ghost commented 6 years ago

Im getting a 'dummy' line, how can i avoid this from happening?

edasque commented 6 years ago

What do you mean, can you run the command and copy out the input and output here?

ghost commented 6 years ago

I run:

node dynamoDBtoCSV.js -t TableName > result.csv

And on the csv i see:

get

But this 'dummy' line isn't on my dynamoDB its added by the code.

edasque commented 6 years ago

@tcchau can you speak to that? Seems that line wasn't unmarshalled correctly, causing dummy rows to be inserted in its place?

tcchau commented 6 years ago

@edasque @MarcoPorracin Hi there. Yes, sorry, I didn't have enough time to solve the original problem completely.

Let me explain: The original problem is that if you have a table that uses a variable set of attributes for items, the first item that is retrieved by dynamoDBtoCSV will define the set of attributes that will be output. That is, if some of your items have an attribute called, 'optionalData', but not the first item, then 'optionalData' from any other item will not be included in the output.

The fastest way for me to resolve this problem was to create a dummy first item that contains every possible attribute that exists in any item in the whole table.

The proper fix would have been to fix the output generation code so that it is aware that we're working with items that may contain additional attributes.

If I have time later on this month, I can work on that, but if someone else can get to it first, by all means!

edasque commented 6 years ago

So @tcchau that line just has to be excluded from the CSV output (or just removed from that doc)?

tcchau commented 6 years ago

Yes, @edasque, I just remove it from my CSV output file manually, since I only occasionally run such a report.

tcchau commented 6 years ago

@MarcoPorracin @edasque It turns out, upon reviewing the papaparse library documentation a little more closely, there's a way to eliminate the need for the dummy row fairly easily. I've issued a pull request to fix this #32

ghost commented 6 years ago

Fixed on pull request #32