jszwec / csvutil

csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.
MIT License
944 stars 62 forks source link

method for normalizing headers #45

Closed meri-burgess closed 3 years ago

meri-burgess commented 3 years ago

It would be great to allow flexibility within headers so that csv files can be reliably unmarshaled into structs with designated tags, despite variance in the way the headers are written. For example, given the struct

type myStruct struct {
      val1 'csv:"val1"'
      val2 'csv:"val2"'
}

and a submitted csv

VAL1,Val2
11,6
4,18
20,21

the submitted csv would not get unmarshaled as one might expect, since the headers are differently cased from those defined in the struct tags

The current work around is to read the a csv body, apply normalization to only the header line, and the re-write it and use that for unmarshaling. Example:

records, _ := csv.NewReader(body).ReadAll()
for i := range records[0] {
    records[0][i] = strings.ToLower(records[0][i])
}
buf := new(bytes.Buffer)
csv.NewWriter(buf).WriteAll(records)

_ = csvutil.Unmarshal(buf.Bytes(), &myStruct)

This is somewhat hefty and inconvenient. What would be more convenient is being able to call a single library function that sets a normalization procedure that could automatically be applied to the header when it is being decoded in the csvutil library, similar to https://pkg.go.dev/github.com/gocarina/gocsv#SetHeaderNormalizer This is just one idea based on other implementations, but other solutions could potentially be applied.

jszwec commented 3 years ago

This is somewhat problematic. How would we solve potential header conflicts? we would probably have to disallow header duplicates while calling NewDecoder