Determine way to speed up calculate_feature_matrix

[gsheni]

• We need to investigate different ways to speed up DFS. This spike encapsulates an investigation, and determining the best way to do this going forward:
• https://github.com/alteryx/featuretools/issues/1913
• https://github.com/alteryx/featuretools/issues/906
• https://www.datarevenue.com/en-blog/pandas-vs-dask-vs-vaex-vs-modin-vs-rapids-vs-ray

[Excidion]

Maybe the issue can be helped by using something like narwhals to encourage users to "bring their own dataframe". This way performance would be mainly down to user choice.

[MarcoGorelli]

thanks @Excidion for your ping in the Narwhals repo, which brought me here!

I'm a fan of featuretools 🤩 (and made a little contribution a few years ago https://github.com/alteryx/featuretools/pull/727), it's exactly the kind of use case I had in mind for Narwhals. Polars really shines for feature engineering, see https://labs.quansight.org/blog/scikit-lego-narwhals for an example of the benefits that Narwhalifying scikit-lego brought

I'd be happy to help out here if you'd be interested, feel free to reach out (here or on Discord, link in the Narwhals readme) with any questions / requests! 🙏