strict: false - headers lost after row with wrong number of columns

aheissenberger commented 4 years ago

Operating System: MacOS 10.14.6
Node Version: v12.11.1
NPM Version: 6.11.3
csv-parser Version: 2.3.1

Expected Behavior

headers created with "mapHeaders" should be used for all valid records, even for records with to many or to less columns

Actual Behavior

when a line contains more columns than headers the broken record is returned with indexed keys.
any valid line following the broken line is returned with indexed key and not with the headers

How Do We Reproduce?

https://github.com/aheissenberger/csv-parser?organization=aheissenberger&organization=aheissenberger

1 test failed

  strictNo › strictNo

  /Users/ah/SVN-Checkouts/DEV/csv-parser/test/strictNo.test.js:9

   8:     const headersLastLine = Object.keys(lines[2]) 
   9:     t.deepEqual(headersFirstLine, headersLastLine)
   10:     t.false(err, 'no err')                        

  Difference:

    [
  -   'a',
  +   '0',
  -   'b',
  +   '1',
  -   'c',
  +   '2',
    ]

armontero commented 4 years ago

Any updates on the fix for this issue? I'm stuck trying to parse a large file and I encounter this error about 2/3 of the way through. All the data after this error only includes the indexed keys.

Is there a workaround in the meantime? Thanks!

armontero commented 4 years ago

It appears that when this happens and the headers are replaced by column indexes, the 'end' event is never generated. I wrote a function to re-apply the headers once this problem is encountered, but now my code hangs when the end of the file is reached. It never enters the code block for the .on('end') event.

ewrayjohnson commented 4 years ago

I am seeing this too

58bcbedf47bd91439c commented 3 years ago

Has this bug been fixed in the version of CSV parser: 2.3.3 When using the version of CSV parser: 2.3.3, this problem still occurs

mafintosh / csv-parser