Open j-grover opened 4 years ago
@j-grover is this an issue with autonormalize or Featuretools? If featuretools, please post as an issue that that repo: https://github.com/featuretools/featuretools/
@kmax12
For reference: autonormalize.py
The normalization of a EntitySet follows the following call graph: normalize_entity -> auto_entityset -> make_entityset
According to my understanding, the variable types are not carried forward from normalize_entity to auto_entityset. So when entities are created in make_entityset, we do not have variable types:
if time_index in current.df.columns:
entities[current.index[0]] = (current.df, current.index[0], time_index)
else:
entities[current.index[0]] = (current.df, current.index[0])
Entities definition:
"""
entities (dict[str -> tuple(pd.DataFrame, str, str)]): Dictionary of
entities. Entries take the format
{entity id -> (dataframe, id column, (time_column), (variable_types))}.
Note that time_column and variable_types are optional.
"""
@j-grover thanks for clarification. I see the issue now.
you're right that we aren't carrying the variable types through. would you be interested in submitting a PR that does that?
@j-grover thanks for clarification. I see the issue now.
you're right that we aren't carrying the variable types through. would you be interested in submitting a PR that does that?
Yeah sure, I'll give it a go.
@kmax12 I have a branch ready, I believe I do not have access to push.
@j-grover can you create a fork to make the pull request?
@j-grover can you create a fork to make the pull request?
Thanks, created PR.
Reproducible example:
Column ip_address is set to dtype featuretools.variable_types.IPAddress:
After normalisation, ip_address resolves back to categorical:
To get the desired features, the variable types need to be preserved so the right primitives can be applied when running dfs. My question is whether this should be the desired behaviour or do the variable types need to be set manually again?