antoinecarme / pyaf

PyAF is an Open Source Python library for Automatic Time Series Forecasting built on top of popular pydata modules.
BSD 3-Clause "New" or "Revised" License
459 stars 72 forks source link

Signal Transformation Computation is too slow #41

Closed antoinecarme closed 7 years ago

antoinecarme commented 7 years ago

We copy the dataset and use a loop over the whole dataset which is too baaaaad :

'''

def specific_invert(self, df):
    df_orig = df.copy();
    df_orig.iloc[0] = df.iloc[0];
    for i in range(1,df.shape[0]):
        df_orig.iloc[i] = df.iloc[i] -  df.iloc[i - 1]
    return df_orig;

'''

There is room for improvement.

antoinecarme commented 7 years ago

Fix : use aggreagtes like cumsum, cumprod on shifted series etc

for the previous example , an equivalent rewrite is :

def specific_invert(self, df):
    df_orig = df - df.shift(1);
    df_orig.iloc[0] = df.iloc[0];
    return df_orig;
antoinecarme commented 7 years ago

detect all such errors in all transformations.