pola-rs / pyo3-polars

Plugins/extension for Polars
MIT License
238 stars 39 forks source link

fix: add `polars` to `requirements.txt` for `derive_expression` example #28

Closed cmdlineluser closed 11 months ago

cmdlineluser commented 11 months ago

I was just experimenting with the new Plugins API example and needed to make a few edits.

make run was failing as polars was missing from requirements.txt so wasn't being installed into the venv

make clean failed due to a typo in the directory name.

The URL for the derive_example in the README.md needed the branch name adjusted. (currently 404)

ritchie46 commented 11 months ago

Thank you. Other than than, were you successful?

cmdlineluser commented 11 months ago

Yes, thanks @ritchie46

It was for a recent "vertical cumfold" question. (similar to https://github.com/pola-rs/polars/issues/7658)

https://stackoverflow.com/questions/77196845/how-to-compute-a-new-polars-column-based-on-its-previous-value

I thought it may be a good opportunity to showcase this new behaviour, so I basically just modified the example you provided: https://github.com/cmdlineluser/polars-plugin-example

It can probably be improved upon with better Rust/Polars-internals knowledge.

Not sure if there is some way to generalize this Iterator::scan() approach for any type.

ritchie46 commented 11 months ago

Not sure if there is some way to generalize this Iterator::scan() approach for any type.

No, you need to specialize it. That's why we don't have a good cumulative story in polars. It is hard to do properly in an interpreted way without paying a huge dynamic dispatch and recompute cost.

Deriving specialized expressions for this is indeed perfect.