yukithm / json2csv

Convert JSON to CSV (go package and command line tool)
MIT License
83 stars 30 forks source link

Seems that WriteCSV is losing some of its data #6

Closed squillace91 closed 4 years ago

squillace91 commented 4 years ago

I have noticed that some conversion (can't figure it out the pattern that is causing it) lose its data when converting the content to the csv file.

Steps to reproduce:

b := &bytes.Buffer{}
wr := json2csv.NewCSVWriter(b)
responses := []map[string]interface{}{} 

//populate responses

csvContent, err := json2csv.JSON2CSV(responses) // csvContent seems to be complete!
wr.WriteCSV(csvContent) 
wr.Flush()
b.Bytes()

At this point when I print the string of the bytes, I get 2 columns missing.

Seems that the issue happens here: image

Which doesn't make sense because the value is there image image

yukithm commented 4 years ago

Thank you for your report.

I have reproduced the problem. It seems that the trailing whitespace of the key has been removed in the process somewhere.

func TestKeyWithTrailingSpace(t *testing.T) {
    b := &bytes.Buffer{}
    wr := json2csv.NewCSVWriter(b)
    responses := []map[string]interface{}{
        {
            " A":  1,
            "B ":  "foo",
            "C  ": "FOO",
        },
        {
            " A":  2,
            "B ":  "bar",
            "C  ": "BAR",
        },
    }

    csvContent, err := json2csv.JSON2CSV(responses) // csvContent seems to be complete!
    if err != nil {
        t.Fatal(err)
    }
    wr.WriteCSV(csvContent)
    wr.Flush()

    got := b.String()
    want := `/ A,/B ,/C  
1,foo,FOO
2,bar,BAR
`

    if got != want {
        t.Errorf("Expected %v, but %v", want, got)
    }
}
--- FAIL: TestKeyWithTrailingSpace (0.00s)
    /Users/yukithm/projects/json2csv/csv_writer_test.go:40: Expected / A,/B ,/C  
        1,foo,FOO
        2,bar,BAR
        , but / A,/B,/C
        1,,
        2,,

B and C are trimmed.

expected keys: "/ A", "/B ", "/C  "
actual keys: "/ A", "/B", "/C"
yukithm commented 4 years ago

Fixed and release v0.1.2.

squillace91 commented 4 years ago

Thanks for the quick fix! Saved me a bunch of time from sanitizing the headers!