Research: how would Narwhals work in scikit-learn?

[MarcoGorelli]

Scikit-learn mentioned Narwhals here https://github.com/scikit-learn/scikit-learn/pull/28513#issuecomment-2131226993

Regardless of whether they decide to use it or not, it would be beneficial to check whether Narwhals would at least work in scikit-learn, and whether it could in principle solve the linked issue

@EdAbati - you've contributed to scikit-learn, and you know Narwhals well - perhaps this issue might be of interest to you?

[EdAbati]

Uuuh this can be fun! I contributed to features related to the Array API compatibility lately. But I'll gladly also try to figure out this part of scikit-learn

it would be beneficial to check whether Narwhals would at least work in scikit-learn, and whether it could in principle solve the linked issue

Agreed, let's start having a look at these:

• [ ] sklearn.inspection.permutation_importance @EdAbati working on it
• [ ] sklearn.model_selection. cross_val_score
• [ ] sklearn.model_selection. cross_validate

If anyone wants to help, please feel free to comment.

[EdAbati]

Some early thoughts:

• what could be the cleanest way to select by row index? the pandas equivalent would be X.iloc[shuffling_idx, col_idx], I guess polars also works with X[shuffling_idx, col_idx], right? what do you think, is it a missing feature?
• what about .copy()/.clone()? FYI I am looking at this

[MarcoGorelli]

what do you think, is it a missing feature?

I'd say so, yes! I think we need this one in Altair too. Does implementing it interest you? I think it just requires an extra branch DataFrame__getitem__

what about .copy()/.clone()?

Sure, doesn't hurt to add DataFrame.clone 👍

[EdAbati]

Does implementing it interest you?

Most of the time my answer to the question is "yes". My only problem is time 😅 I will create an issue in case someone else is able to pick it up before me

[MarcoGorelli]

😄 I'll take the getitem one on then so we can propose it to Altair too