search

antoinecarme / pyaf

PyAF is an Open Source Python library for Automatic Time Series Forecasting built on top of popular pydata modules.
BSD 3-Clause "New" or "Revised" License
459 stars 72 forks source link

Update Model Complexity Computations #171

Closed antoinecarme closed 3 years ago

antoinecarme commented 3 years ago

PyAF uses a notion of complexity for each component of the model (signal transformation, trend, cycle.seasonal, AR, ... )

The complexity value is used when two models have almost the same performance (MAPE) , the less complex model is kept in this case (Occam's razor).

By introducing new models (Croston, XGBoost, LightGBM), the notion of complexity has to be dependent of each component and the overall weighting is no longer relevant.

this change will not impact too much the output of PyAF as the complexity is not always used in the model selection.

antoinecarme commented 3 years ago 

~/dev/python/packages/timeseries/pyaf$ grep mComplexity pyaf/TS/SignalDecomposition*.py
pyaf/TS/SignalDecomposition_AR.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_AR.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 1;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 2;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 2;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 3;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 3;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 1;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 1;

~/dev/python/packages/timeseries/pyaf$ grep mComplexity pyaf/TS/Scikit_Models*.py
        self.mComplexity = len(self.mInputNamesAfterSelection)
        self.mComplexity = P;
        self.mComplexity = 2*P;
        self.mComplexity = 2*P;
        self.mComplexity = 2*P;
antoinecarme commented 3 years ago

Overall weighting (weights should be 1, each model uses its own weight and defines its own complexity)

     def getComplexity(self):
-        lComplexity = 32 * self.mTransformation.mComplexity +  16 * self.mTrend.mComplexity + 4 * self.mCycle.mComplexity + 1 * self.mAR.mComplexity;
antoinecarme commented 3 years ago

New values 

~/dev/python/packages/timeseries/pyaf$ grep mComplexity pyaf/TS/SignalDecomposition*.py
pyaf/TS/SignalDecomposition_AR.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_AR.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = len(self.mEncodedValueDict.keys())
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 0
pyaf/TS/SignalDecomposition_Cycle.py:            self.mComplexity = len(lDict.keys())
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 2;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = iWindow;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = iWindow;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 1;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 3;
antoinecarme commented 3 years ago

New weighting

    def getComplexity(self):
        lComplexity = self.mTransformation.mComplexity +  self.mTrend.mComplexity + self.mCycle.mComplexity + self.mAR.mComplexity;
        return lComplexity;