search

dashbitco / nimble_csv

A simple and fast CSV parsing and dumping library for Elixir
https://hexdocs.pm/nimble_csv
767 stars 51 forks source link

Specific encoding and bom support #46

Closed LostKobrakai closed 4 years ago

LostKobrakai commented 4 years ago

In regards to #45

josevalim commented 4 years ago

Hi @LostKobrakai!

Sorry, I am afraid I was not clear in the first comment. I think the best way to tackle this is to change the binary matching syntax and code to work on UTF-16 upfront, instead of forcing a pass on everything. Also note that the :bom flag is already support on File.stream!, so we don't need to handle it here. The correct is if users pass the flag to File.stream! itself. We should handle bom only on full strings (and handle it only once).

On dumping, however, we will have to do a post pass after all of the data has been encoded.

josevalim commented 4 years ago

Actually, let me play with these ideas a bit. I will follow up later today. Thanks!

LostKobrakai commented 4 years ago

To be honest, the dumping part is the one I'm most interested in, so feel free to change how parsing is handled.

josevalim commented 4 years ago

:green_heart: :blue_heart: :purple_heart: :yellow_heart: :heart: