search

ibis-project / ibis

the portable Python dataframe library
https://ibis-project.org
Apache License 2.0
4.36k stars 540 forks source link

feat(polars): support `to_torch` #9159

Open ianmcook opened 3 weeks ago

ianmcook commented 3 weeks ago

I see Polars recently added to_torch: https://www.linkedin.com/posts/pola-rs_export-your-polars-dataframe-directly-to-activity-7193568311769542656-kqRz

I see Ibis has a to_torch API: https://github.com/ibis-project/ibis/pull/6518

Is the former supported via the latter?

Code of Conduct

gforsyth commented 3 weeks ago

It is not, currently.

We go from any backend (including polars) -> pyarrow -> numpy -> torch.from_numpy

Polars has it's own fast (er?) path from a polars frame to a numpy array, then also uses torch.from_numpy.

ianmcook commented 3 weeks ago

Got it, thanks. Yeah, I don't know if it's worthwhile to directly map Ibis to_torch to Polars to_torch for performance or other reasons.