Add quoted information to CSV::FieldInfo #252

[heronshoes]

This is a proposal to fix the issue #252;

• Add a new member :quoted? to FieldInfo .
• Introduce an array of booleans @quoted_fields in Parser to collect original column value was quoted?.
• Check in each of parse_no_quote, parse_quotable_loose and parse_quotable_robust.
  • Use local variable quoted_fields to collect booleans in each method and pass it through #emit_row.
• Add test of custom converter for preserving quoted value.

[heronshoes]

I rebased feat/issue-252 to the latest HEAD.

[heronshoes]

Thanks! I will check. (I think this style is better because I can check one by one.)

[heronshoes]

I will add test case in:

• "quoted",unquoted,"quoted-again"
• Put expected @str in assertion locally
• CSV.parse(..., headers: '"serial",value', ...) (specifying headers as String) case

Please hold approving.

[heronshoes]

I added and fixed except 3rd one:

• CSV.parse(..., headers: '"serial",value', ...) (specifying headers as String) case

This one requires fix in prepare_header, parse_headers of parser.rb, but I can't fix yet. It may also require Symbol#encode fix.

Current CSV has a bug in converting when specified header has a Symbol.

CSV.parse("1,2", headers: ["string", :symbol], header_converters: :symbol)

CSV.parse(..., headers: '"serial",value', ...) case requires prior parsing, so same issue will happen.

[kou]

How about this?

diff --git a/lib/csv/parser.rb b/lib/csv/parser.rb
index c3c1eff..cd5a45f 100644
--- a/lib/csv/parser.rb
+++ b/lib/csv/parser.rb
@@ -758,9 +758,10 @@ class CSV
       case headers
       when Array
         @raw_headers = headers
+        quoted_fields = [false] * @raw_headers.size
         @use_headers = true
       when String
-        @raw_headers = parse_headers(headers)
+        @raw_headers, quoted_fields = parse_headers(headers)
         @use_headers = true
       when nil, false
         @raw_headers = nil
@@ -770,20 +771,24 @@ class CSV
         @use_headers = true
       end
       if @raw_headers
-        # Set 'quoted?' info to false to avoid re-converting
-        @headers = adjust_headers(@raw_headers, [true] * @raw_headers.size)
+        @headers = adjust_headers(@raw_headers, quoted_fields)
       else
         @headers = nil
       end
     end

     def parse_headers(row)
-      converters = @return_headers ? [] : @header_fields_converter.to_a
-      CSV.parse_line(row,
-                     col_sep:    @column_separator,
-                     row_sep:    @row_separator,
-                     quote_char: @quote_character,
-                     converters: converters)
+      quoted_fields = []
+      converter = lambda do |field, info|
+        quoted_fields << info.quoted?
+        field
+      end
+      headers = CSV.parse_line(row,
+                               col_sep:    @column_separator,
+                               row_sep:    @row_separator,
+                               quote_char: @quote_character,
+                               converters: [converter])
+      [headers, quoted_fields]
     end

     def adjust_headers(headers, quoted_fields)

[heronshoes]

Thank you!! I reflected this code!

[kou]

Thanks!