ImperialCollegeLondon / pycsvy

Python reader/writer for CSV files with YAML header information
BSD 3-Clause "New" or "Revised" License
6 stars 7 forks source link

Add support for `polars` #83

Closed dalonsoa closed 3 days ago

dalonsoa commented 3 days ago

Problem

polars has become recently a very attractive - and faster - alternative to pandas, so we should support it, as well.

Solution

Largely copying what's in place for pandas, implement support for reading data into a polars dataframe with read_csv and from a polars dataframe with write_csv.

AdrianDAlessandro commented 3 days ago

I'll take this. I believe the best way to do this is to start using narwhals (which provides a dataframe-agnostic API). There doesn't appear to be support for a read_csv in Narwhals, but there is a write_csv that we can use.

dalonsoa commented 3 days ago

If there's no read_csv, how are you planning to proceed for creating polars dataframes?

AdrianDAlessandro commented 3 days ago

I will implement it using the read_csv from polars. This will probably be two separate PRs, one for read and one for write.

dalonsoa commented 3 days ago

But that will add two dependencies rather than just one, polars. Isn't it a bit overkilling to use narwhals in this case, specially considering that it only implements half of what we need?

AdrianDAlessandro commented 3 days ago

I'll do it just with polars for now and open a separate issue about narwhals