search

alteryx / featuretools

An open source python library for automated feature engineering
https://www.featuretools.com
BSD 3-Clause "New" or "Revised" License
7.26k stars 879 forks source link

Remove extra if block in calculate_chunk that was added due to Koalas bug #1647

Open thehomebrewnerd opened 3 years ago

thehomebrewnerd commented 3 years ago

Remove extra if block in calculate_chunk that was added due to Koalas bug

During the Woodwork integration, a code block was added to address an issue where appending Koalas dataframes could cause the Woodwork schema to be invalidated as a result of Koalas changing the dtypes of the columns unexpectedly.

Koalas Issue 2193 was created to resolve this issue. Once the Koalas issue is closed and a fix is released, we should be able to delete the code block shown below from the calculate_chunk method in calculate_feature_matrix.py.

        if not fm.ww.schema:
            # Koalas bug sometimes causes schema to be invalidated
            # after `append` so need to reinitialize WW
            fm.ww.init(**ww_init_kwargs)
gsheni commented 2 years ago

Still blocked on https://github.com/databricks/koalas/issues/2193