search

gmo / common

MIT License
0 stars 0 forks source link

CsvFile abstraction #8

Closed CarsonF closed 6 years ago

CarsonF commented 6 years ago

An adaption of what we've done in our other libraries/systems to read CSV files with the headers mapped to the rows values. This one is based on SplFileObject which has a lot of the CSV logic builtin. Unfortunately, some of it is not handled very well so that's remedied here. The actual iteration is more robust here as well, with better error handling and the ability to getHeaders and manually set them (in case the file doesn't have them).

This also works with CSV files with mac line endings \r using PHP's builtin auto detection. This throws an exception in our other implementations.

CarsonF commented 6 years ago

@mikegillis677 Can you look over those additions for writing? I wanted to implement some kind of "auto header" writing but couldn't figure out a good way to do it. Plus it was already complex with headers being able to be read from file, keys from first row written, or set manually.

I think what I have here covers all of our use cases. I don't think redshift CSVs have headers, but maybe I can't remember correctly. Those are the only ones we write to. The geo ones are only read from so they should already be good from the first commit.

mikegillis677 commented 6 years ago

The changes look sane. I checked, and Redshift CSVs do not have headers.