jehugaleahsa / FlatFiles

Reads and writes CSV, fixed-length and other flat file formats with a focus on schema definition, configuration and speed.
The Unlicense
357 stars 64 forks source link

Bug in RecordError functionality when IsHandled is set #81

Closed kamatvedita99 closed 2 years ago

kamatvedita99 commented 2 years ago

My business case I want to parse a fixed length file, and collect all parsing errors in a data structure. For FixedLengthTypeMapper, I am using the event RecordError and setting IsHandled to true. The expected behaviour is that it parses all rows and gathers all exceptions in the data structure. But it stops parsing after first erraneous record is encountered. Source Code image

image File Content image

Output image

Expected Output Record: 123Bob17Exception: FlatFiles.RecordProcessingException: The record did not meet the length requirements specified in the schema. Record 1. Record: D24Jorw233Exception: FlatFiles.RecordProcessingException: Encountered an invalid conversion while processing the record. Record 2. Version: I am using FlatFiles 4.16.0 version installed via Nuget Package Manager, .NET framework - 4.7.2 Additional context I have tried running this by cloning your source code, and it works absolutely fine i.e gathers all exceptions. But when installed via Nuget Package Manager it does not work. Also, as per my observation it does not work when the error is related to row length not matching schema requirements.It works as expected if the errors are column parsing errors.

jehugaleahsa commented 2 years ago

Sorry if it's not related, but your code sample doesn't make sense. You are building a single string containing the three data rows and then calling String.Join on it. You probably want:

string[] lines = new string[] 
{
    "123Bob17",
    "D24Tomw233",
    "123Ved1714"
};
kamatvedita99 commented 2 years ago

@jehugaleahsa You are right. I was actually reading from file, so the lines are stored as IEnumerable of string type. Hence I am calling String.Join on it. I have modified the example. Hope the issue makes sense now. Let me know if you need any further clarification regarding the same. Attaching the code files for your reference. Console solution : ReadingFile.zip Input file : test.txt

jehugaleahsa commented 2 years ago

I decided to just move forward with my changes in master and created a 5.0.0 release: https://www.nuget.org/packages/FlatFiles/

There may be some incompatibilities with the 4.x version. Primarily, I renamed the SeparatedValue* classes to Delimited*, but there might have been a few other subtle differences.

kamatvedita99 commented 2 years ago

Thanks Travis. This issue is resolved in 5.0.0 release.

kamatvedita99 commented 2 years ago

If you don't mind,could you let me know what the real problem was. I am just curious to know, why it wasn't working as expected previously.

jehugaleahsa commented 2 years ago

The old code would see IsHandled set to true and just return null. Other code also used null to indicate the end of file had been reached. It was misinterpreting the null the wrong way and exited immediately.

kamatvedita99 commented 2 years ago

Got you, thanks 😃