Closed HeuristicLab-Trac-Bot closed 11 years ago
Worked on symbolic regression classes to prepare for time series prognosis plugin with r4022.
Moved code for calculation of covariance from the scaled MSE evaluator into a separate online evaluator with r4027.
r6802: added classes (problem, evaluators, analyzers, solution, model, online-calculators, and views) for time series prognosis problems and added an algorithm implementation to generation linear AR (auto-regressive) time series prognosis solution.
- added configuration of default grammar for time-series prognosis
- improved multiple update of variable impacts result
- cleared up definition of accuracy metrics for time series prognosis to make the distinction between one n-step forecast and n one-step forecasts clearer
- implemented calculators for time series accuracy metrics to support the calculation of the average accuracy over m n-step forecasts,
- and adapted the unit tests accordingly.
Open tasks:
- implement unit tests for n-step forecasts (only one-step forecasts are tested right now)
- write blog post / wiki page to document the accuracy metrics
- add a benchmark test set for time series prognosis with financial data
r7058: added unit tests for directional symmetry and Theil's U and multi-step forecasts
A larger overhaul of the time series implementation is necessary.
The most important aspect of time series analysis is prognosis of future values. As of now we only support one-step forecasts from y(t) to y(t+1). However, often it is necessary to create a forecast for the next n time-steps. For uni-variate time series, it would be possible to calculate the forecast outside of HeuristicLab, by simple recurrence (using the first one-step prognosis y(t+1) as additional input for the next one-step prognosis y(t+2) and so on). It would be nice to support this also directly in HeuristicLab and show for instance the line chart of the forecast for the whole test partition.
The second problem when n-step forecasts are supported arises when additional variables are used as inputs for the time-series model. Because the values of the input variables are usually not known for the forecast period. This issue can be solved by learning a multi-variate time-series model instead.
Because of the considerations mentioned above I decided to pull the time-series plugins from the 3.3.6 release as this would lead to difficulties when we have to maintain compatibility in future releases.
The time-series prognosis functionality will be released with 3.3.7.
r7098: created a branch for the development of time series prognosis features.
r7120: implemented multi-variate symbolic expression tree interpreter for time series prognosis.
- implemented remaining metrics for time series prognosis solutions
- added estimation limits
- fixed training and validation best solution analyzers
- implemented overfitting analyzer.
r7194: small changes in calculation of time series solution results
r7461: fixed a few problems related to a previous incorrect merge and removed unrelated projects from the solution.
In the simplified for time series solutions the correct horizon should be used to calculate impacts.
r7463: improved formatter, line chart view, simplifier for time series prognosis solutions
r7843: fixed compile errors after merging changes from the trunk
r7886: Adapted timeseries branch to contain only modified plugins.
r7889: Branched problem instances to enable CSV import for timeseries problems.
Issue migrated from trac ticket # 1081
milestone: HeuristicLab 3.3.8 | component: Problems.DataAnalysis | priority: medium | resolution: done
2010-07-09 17:00:44: @gkronber created the issue