search

yellowfeather / DbfDataReader

DbfDataReader is a small fast .Net Core library for reading dBase, xBase, Clipper and FoxPro database files
MIT License
134 stars 61 forks source link

DbfRecord Read reading EOF in middle of stream. #101

Closed CryoMyst closed 1 year ago

CryoMyst commented 3 years ago

We are getting cases where it will stop reading a DBF halfway through so the extracted row count does not equal what the header says.

(Examples, table names removed) Table1 Difference in row count got 273135(0 marked deleted) expected 770187 Table2 Difference in row count got 792885(0 marked deleted) expected 1047742 Table3 Difference in row count got 86359(0 marked deleted) expected 121140 Table4 Difference in row count got 604003(0 marked deleted) expected 885758 Table5 Difference in row count got 296093(0 marked deleted) expected 937534 Table6 Difference in row count got 31434(240 marked deleted) expected 52718 Table7 Difference in row count got 691370(116440 marked deleted) expected 1017245

To Reproduce Do not have any reproduction steps, this has only happened on 2 sites and the data itself is sensitive.

Expected behavior The DBF should detect it has not read all entries and continue reading.

Desktop (please complete the following information):

Additional context DbfRecord.cs to also check the stream position is the stream length and skipping forward returning a null record if not. And this seems to read all entries now however I am not sure this is the proper way as I do not know the file spec.

chrisrichards commented 3 years ago

Is this still an issue? Do you have any more information?

CryoMyst commented 3 years ago

I am unsure what exactly is happening, just that DbdRecord is hitting EndOfFile at the beginning of the record. I have confirmed that even OleDB adapter would fail to read records past this.

My temp fix was to change if (value == EndOfFile) return false; to if (value == EndOfFile && binaryReader.BaseStream.Position == binaryReader.BaseStream.Length) return false;

This allowed for reading of the whole Dbf file which only that one field corrupted. This should probably be changed to setting the offset for each record since each field is a fixed length and allowing for single records to be "unreadable"

(I have also run into another issue where in DbfMemoFoxPro.BuildMemo value is null which also crashes program but that might be unrelated )

CryoMyst commented 1 year ago

I am going to say this was due to dbf corruption over anything here. Closing.