search

Clever / optimus

Concurrently extract, transform, and load tables of data in Go
Apache License 2.0
34 stars 7 forks source link

Support configurable csv delimiters #12

Closed kvigen closed 9 years ago

kvigen commented 9 years ago

Previously, the csv source didn't allow configuration of CSV options and would always use the defaults. This means that lots of things were impossible, e.g. reading a tab-delimited file.

This change maintains the old function for backwards compatibility, but adds a new function that creates a csv source from a csv.Reader, allowing the consumer to specify any options they desire.

We test this by making an io.Reader with tab-delimited CSV data, making a csv source from it, and verifying that we get the correct data from the Table.

kvigen commented 9 years ago

@azylman assigned to you

azylman commented 9 years ago

What do you think of, instead of adding functions that support custom delimiters, adding functions that take in arbitrary csv.Readers and csv.Writers? Then people can use whatever options they want.

kvigen commented 9 years ago

Agreed https://github.com/Clever/optimus/commit/d9f94d75bc8c49d25a388d6a8913e4594722a950

azylman commented 9 years ago

This looks great!

azylman commented 9 years ago

lgtm