search

roskakori / cutplace

validate data stored in CSV, PRN, ODS or Excel files
http://cutplace.readthedocs.org/
GNU Lesser General Public License v3.0
18 stars 20 forks source link

Fails when last char of field is escaped #49

Closed ghost closed 9 years ago

ghost commented 11 years ago

OpenSUSE 12.1, Python 2.7.2, cutplace-0.7.1

Using the following ICD:

d,format,delimited d,line delimiter,lf d,encoding,ascii d,quote character,"""" d,escape character,"\" f,TEST1,,,,Text,

This CSV passes...

"\"A"

... but this one fails: [[[ looks like it processes an additional char (the final "), because "A\""" passes ]]]

"A\""

ERROR:cutplace:cannot handle unexpected error: newline inside string Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/cutplace-0.7.1-py2.7.egg/cutplace/_cutplace.py", line 308, in main result = process(argv) File "/usr/local/lib/python2.7/site-packages/cutplace-0.7.1-py2.7.egg/cutplace/_cutplace.py", line 285, in process cutPlace.validate(path) File "/usr/local/lib/python2.7/site-packages/cutplace-0.7.1-py2.7.egg/cutplace/_cutplace.py", line 215, in validate self.icd.validate(dataFilePath) File "/usr/local/lib/python2.7/site-packages/cutplace-0.7.1-py2.7.egg/cutplace/interface.py", line 709, in validate for row in reader: File "/usr/local/lib/python2.7/site-packages/cutplace-0.7.1-py2.7.egg/cutplace/_parsers.py", line 174, in delimitedReader producer.join() File "/usr/local/lib/python2.7/site-packages/cutplace-0.7.1-py2.7.egg/cutplace/_parsers.py", line 88, in join raise self._error Error: newline inside string

roskakori commented 9 years ago

This should be fixed with the current code base, but we still need a test case for it.

roskakori commented 9 years ago

(brainfart; "" is CSV escaped ")

roskakori commented 9 years ago

Fixed and guarded by a proper test case.

Thanks for the clean and compact example, which pointed out (a different) bug in the reimplementation with Python 3. There is some quite complex interaction between the quotechar, escapechar, doublequote and probably also strict attributes of the csv.reader in the standard library.