phatcher / CsvReader

Extended version of Sebastian Lorien's fast CSV Reader
MIT License
300 stars 102 forks source link

CsvReader throws MalformedCsvException when only some columns are mapped to SqlBulkCopy #28

Open isolaol opened 8 years ago

isolaol commented 8 years ago

Hi,

The behavior of the CsvReader and CachedCsvReader differ from each other when only some columns are mapped to SqlBulkCopy column collection. The CsvReader class throws MalformedCsvException while CachedCsvReader works as expected. The following (simplified from real project) code snippets should reproduce the behavior:

// throws MalformedCsvException at character position 2092
using (CsvReader parser = new CsvReader(new StreamReader(@"C:\Temp\Sample.txt"), true, '\t')) {
    using (SqlBulkCopy loader = new SqlBulkCopy("...")) {
        loader.DestinationTableName = "Sample";
        loader.ColumnMappings.Add("ColA", "ColA");
        loader.WriteToServer(parser);
    }
}

// works
using (CsvReader parser = new CachedCsvReader(new StreamReader(@"C:\Temp\Sample.txt"), true, '\t')) {
    using (SqlBulkCopy loader = new SqlBulkCopy("...")) {
        loader.DestinationTableName = "Sample";
        loader.ColumnMappings.Add("ColA", "ColA");
        loader.WriteToServer(parser);
    }
}

Is this a bug or by design?

The sample input file contains two tab-separated text columns (ColA, ColB) with headers. The sample table contains single text column (ColA).

CsvReader and SqlBulkCopy is expected to pump the content of the first column (ColA) from the input file to the table.

Kind regards, Olli-Pekka Isola

phatcher commented 7 years ago

It sounds like a bug - ISTR that SqlBulkCopy was rather strict on the DataTable it will handle e.g. column names and data types have to be aligned

Happy to take a pull-request with a fix