Fixed offset error on last line of chunk (MLDB-2167)

mldbai / mldb

MLDB is the Machine Learning Database

http://mldb.ai

Apache License 2.0

666 stars 102 forks source link

Closed jeremybarnes closed 7 years ago

jeremybarnes commented 7 years ago

Fixed error using gcount() after readline(), leading to thinking that the chunk is bigger than it is and junk characters in between chunks.

mailletf commented 7 years ago

Is there a small test that we can add?

jeremybarnes commented 7 years ago

It's not easy to reproduce, and I only managed to do so using customer data. I'll look to see if I can add a hook to help make testing easier.

jeremybarnes commented 7 years ago

We need at least 20MB of data in order to reproduce in a test case.