How can we handle few missing time indices in featuretools? Even one missing timestamp in the time index leads to all time based features being null #2700
Many times it so happens that you do not have the time index of a data entry and a lot of other features are present. However, the time index is still present in a lot of the cases and thus the time based features could and should be computed.
I am not finding a way to create such features using featuretools. All the created features are NaN
Example:
from datetime import timedelta
import featuretools as ft
import pandas as pd
import numpy as np
from featuretools import Timedelta
tw = Timedelta(3, unit = "d")
ts = pd.to_datetime("01-01-2020 01:00:00")
time_index = list(pd.date_range(ts, ts + timedelta(hours = 168), periods = 8, inclusive = 'both'))
time_index[4] = np.nan#np.datetime64("NaT")
df1 = pd.DataFrame({'ind': [1], 'time': ts})
df2 = pd.DataFrame({'ind':[1,2,3,4,5,6,7,8], 'id':[1,1,1,1,1,1,1,1], 'time': time_index, 'feat': [np.nan,1,2,4,8,16,48,144]})
es = ft.EntitySet('es')
es.add_dataframe(df1, index = 'ind', dataframe_name = 'base', time_index = 'time')
es.add_dataframe(df2, index = 'ind', dataframe_name = 'data', time_index = 'time')
es.add_relationship('base','ind','data','id')
ct = pd.DataFrame({"instance_id":[1,1,1], "time":[time_index[-1], time_index[-2], time_index[-3]]})
es.add_last_time_indexes()
ft.dfs(entityset = es, target_dataframe_name = 'base', agg_primitives=['trend'],
trans_primitives=[], cutoff_time=ct, cutoff_time_in_index=True, training_window = tw)[0].sort_index()
The output is all Null features. However, if I comment out the one line setting one of the time indices to 0 (time_index[4] = np.nan#np.datetime64("NaT")) then, we can see that the features are getting generated just fine.
How can we handle missing time indices in featuretools?
Many times it so happens that you do not have the time index of a data entry and a lot of other features are present. However, the time index is still present in a lot of the cases and thus the time based features could and should be computed.
I am not finding a way to create such features using featuretools. All the created features are NaN
Example:
The output is all Null features. However, if I comment out the one line setting one of the time indices to 0 (
time_index[4] = np.nan#np.datetime64("NaT")) then, we can see that the features are getting generated just fine.How can we handle missing time indices in featuretools?
Output of
featuretools.show_info()