lvgig / tubular

Python package implementing transformers for pre processing steps for machine learning.
https://tubular.readthedocs.io/en/latest/index.html
BSD 3-Clause "New" or "Revised" License
38 stars 14 forks source link

Narwhalify BaseNominalTransformer #334

Open limlam96 opened 4 days ago

limlam96 commented 4 days ago

What

make BaseNominalTransformer polars compatible while maintaining backwards compatibility with pandas.

Why?

Making the project polars compatible for speed benefits. Narwhals allows us to do this piece by piece and maintain compatibility between components as it makes it possible to pass pandas dataframes to polars logic. This involves converting between pandas/polars dataframes.

Without converting parent classes there will be lots of switching back and forth due to calls to super() so we should aim to convert foundational elements early on.

How?

convert logic to function with polars dataframes and add narwhalify decorators to methods to ensure compatibility with pandas dataframes.

Update testing where required. This will require additional tests or parameterisation for polars workflows.

limlam96 commented 3 days ago

https://github.com/lvgig/tubular/pull/335