fixed potential error with incorrect if statement. the tokenizer is now used for every line parsing process.
The original statement
if (options.Delimiter.Equals(options.Separator))
dataValues = Tokenize(sLine, options.Delimiter); //use custom tokenizer for improved performance
else
dataValues = sLine.Split(options.Delimiter); //use ordinary Split function for simple cases
was always false since otherwise it would've been impossible to corretcly read the csv file. now it always uses the Tokenizer and throws an ArgumentException if delimiter and separator are equal.
fixed potential error with incorrect if statement. the tokenizer is now used for every line parsing process.
The original statement
was always false since otherwise it would've been impossible to corretcly read the csv file. now it always uses the Tokenizer and throws an ArgumentException if delimiter and separator are equal.