evanplaice
commented
9 years ago From evanpla...@gmail.com on October 10, 2012 15:06:51
Could you provide a data sample stub that causes the failure.
From a technical standpoint, even empty lines should contain delimiters to represent the number of columns in the data.
To quote RFC 4180:
-
Within the header and each record, there may be one or more fields, separated by commas. Each line should contain the same number of fields throughout the file. Spaces are considered part of a field and should not be ignored. The last field in the record must not be followed by a comma. For example:
aaa,bbb,ccc
Only the last line is allowed to be completely empty in a valid CSV file and the parser should handle it properly if one exists.
So an empty line with 3 columns would still show up as:
,,Basically, a CSV file must have the same number of columns in every row to be considered valid. Even if the trailing values on each row are consistently empty.
There are two potential solutions to this problem...
First, you could pre-process the line data. A hook called onParseValue() exists to allow users to insert an arbitrary function to do additional processing on CSV values. To give users better fine-tuned control over the parser I was planning on including another called onParseEntry() that does the same but on a full line of data.
With that in place you will be able to insert a function that adds in the correct number of delimiters (ex commas).
The second solution works the same as you have already done. Do a full pass on the data first to remove empty lines. I'll be adding two hooks, onPreParse() onPostParse() to do the same.
There are two schools of thought on this. Those who want to liberate the data by making the parser more versatile and those who think the users should be held responsible for providing 'clean' data.
To see a bad example of the too-liberal approach, just look at pre-HTML 4.01 browsers and 'quirks mode'. Too much variability in the data requires a lot of 'magic' (read 'hacks') to exist below the surface.
CSV already a very liberal format that allows a lot of funky edge cases that need to be covered in the parser. My main overarching goal is to provide a parser that specializes in CSV data and not necessarily anything beyond CSV.
There are a LOT of use cases where users want to have the ability to extend the parser to do more interesting things. I don't want to disregard that request altogether so my approach is to provide hooks. By doing so users have the ability insert custom code inline during the different phases of the parsing process and add their own domain-specific magic where it's needed.
Let me know what you think...
From Chad.R.B...@gmail.com on October 10, 2012 13:42:48
What steps will reproduce the problem? 1. Pass a CSV file with one or more blank lines into $.csv.toObjects(file) What is the expected output? What do you see instead? I expect it to just ignore the blank lines, or do something with them. At least a blank line at the end. I would expect that an extra blank line at the end of CSV files is a common occurrence.
Specifically, it errors here, at line 365.
for(var j in headers) { object[headers[j]] = entry[j]; }
Uncaught TypeError: Cannot read property '0' of undefined
This is because in the case a blank line, entry is undefined. What version of the product are you using? On what operating system? 0.64 on Windows. Please provide any additional information below. It's up to you to decide what your program should do in this case. I figured it out and I can remove the blank lines from my input, but if one of my users puts a blank line in their input, it will just silently fail. I would prefer that to not happen. I guess alternatively, I could check the input before putting it through your function and removing the blank line myself.
Original issue: http://code.google.com/p/jquery-csv/issues/detail?id=13