search

alteryx / woodwork

Woodwork is a Python library that provides robust methods for managing and communicating data typing information.
https://woodwork.alteryx.com
BSD 3-Clause "New" or "Revised" License
145 stars 20 forks source link

Deprecate support for Spark DataFrames #1686

Closed gsheni closed 1 year ago

gsheni commented 1 year ago
ulc0 commented 1 year ago

We are using woodwork and featuretools. I would be happy to contribute if spark deprecation is related to that

gsheni commented 1 year ago

Hi @ulc0, can you describe how you are using Spark DataFrames? How well does it work for you (speed, runtime improvements)? Would you be open to using Dask DataFrames instead?

gsheni commented 1 year ago

Due to the community feedback, we have decided not to remove Spark support in Featuretools

ulc0 commented 1 year ago

Databricks Spark Frames and it works wonderfully. Federal Government Unit. I'll keep my fork going.