Switch to polars over pandas

UK-IPOP / open-data-pipeline

A pipeline for processing, enhancing, and sharing open datasets.

https://uk-ipop.github.io/open-data-pipeline/

GNU General Public License v3.0

1 stars 0 forks source link

Closed nanthony007 closed 1 year ago

nanthony007 commented 1 year ago

Using polars would improve data frame I/O especially with respect to reading/writing jsonlines.

We could then also remove the orjson dependency.

I think it would be best to do this after #13 is resolved.