Refactor Constants Optimization

[HeuristicLab-Trac-Bot]

Issue migrated from trac ticket # 2974

milestone: HeuristicLab 3.3.17 | component: Problems.DataAnalysis.Symbolic.Regression | priority: medium

2018-12-27 12:32:58: @mkommend created the issue

---

The code for constants optimization has grown over the years and all new functionality has been directly added in the evaluator. This should be refactored into separate classes and improvements to its functionality should be implemented. In detail the following changes should be realized:

• Optimizing constants should be separated from evaluation
• Constant optimization should use the MSE instead of the R²
• Add interfaces and static utility methods
• ...

[HeuristicLab-Trac-Bot]

2018-12-27 14:15:28: @mkommend commented

---

r16455: Created branch and added adapted project HL.Problems.DataAnalysis.Symbolic.Regression.

[HeuristicLab-Trac-Bot]

2018-12-27 14:23:51: @mkommend commented

---

r16456: Added HL.Problems.DataAnalysis.Symbolic to the branch.

[HeuristicLab-Trac-Bot]

2018-12-28 10:21:21: @mkommend commented

---

r16457: Extracted linear scaling terms in auto diff converter. \ r16458: Readded accidently remove readonly modifier in autodiff converter.

[HeuristicLab-Trac-Bot]

2018-12-28 10:22:14: @mkommend commented

---

r16459: Added utility methods and copy of constant optimization evluator for refactoring. \ r16460: Adapted project files.

[HeuristicLab-Trac-Bot]

2018-12-28 10:22:14: @mkommend

[HeuristicLab-Trac-Bot]

2018-12-28 17:10:39: @mkommend commented

---

r16461: Added unit tests and refactoring.

[HeuristicLab-Trac-Bot]

2018-12-28 17:57:48: @mkommend commented

---

r16463: Adapted tree to autodiff converter. \ r16464: Adapted new constants optimizer.

[HeuristicLab-Trac-Bot]

2018-12-28 17:57:48: @mkommend

[HeuristicLab-Trac-Bot]

2019-01-04 17:37:27: @mkommend commented

---

r16500: Added intermediate version of new constants optimization for profiling.

[HeuristicLab-Trac-Bot]

2019-01-06 18:04:07: @mkommend commented

---

r16507: First stable version of new CoOp. \ r16508: Reverted changes in TreeToAutoDiffTermConverter.

[HeuristicLab-Trac-Bot]

2019-01-06 23:35:00: @mkommend commented

---

r16509: Added new evaluator with constants optimization.

[HeuristicLab-Trac-Bot]

2019-01-07 18:17:55: @mkommend commented

---

r16514: Updated CoOp to handle negative R² values approprietly.

[HeuristicLab-Trac-Bot]

2019-01-08 17:54:43: @mkommend commented

---

r16522: Fixed bug in constants extraction when adding linear scaling coefficients.

[HeuristicLab-Trac-Bot]

2019-01-10 11:53:19: @foolnotion commented

---

r16525: Add benchmarks solution for testing constant optimization performance.

[HeuristicLab-Trac-Bot]

2019-03-13 09:30:37: @gkronber commented

---

r16676: merged r16478 to r16672 from trunk:HeuristicLab.Problems.DataAnalysis.Symbolic to branch:HeuristicLab.Problems.DataAnalysis.Symbolic

[HeuristicLab-Trac-Bot]

2019-03-13 09:32:37: @gkronber commented

---

r16677: merged r16499 to r16658 from trunk:HeuristicLab.Problems.DataAnalysis.Symbolic.Regression to branch:HeuristicLab.Problems.DataAnalysis.Symbolic.Regression

[HeuristicLab-Trac-Bot]

2019-03-13 09:36:11: @gkronber commented

---

r16678: removed project references again after merging changes from trunk

[HeuristicLab-Trac-Bot]

2019-03-13 09:37:12: @gkronber commented

---

r16679: changed .NET framework version of UnitTests project for compatibility with HL trunk (4.6.1)

[HeuristicLab-Trac-Bot]

2019-03-13 09:38:23: @gkronber commented

---

r16680: necessary changes for compatibility with new persistence

[HeuristicLab-Trac-Bot]

2019-03-16 12:50:11: @gkronber commented

---

r16689: changed LMConstantsOptimizer to use minlm interface of alglib instead of lsfit interface.

[HeuristicLab-Trac-Bot]

2019-03-16 12:51:47: @gkronber commented

---

r16690: added code in the unit tests to make behaviour of new and old ConstOpt consistent to allow direct comparison of results (+fixed a small bug in the unit tests)

[HeuristicLab-Trac-Bot]

2019-03-16 13:14:19: @gkronber commented

---

Currently, one unit test fails because of code in OnlinePearsonRCalculator.

It would work if we would replace it with

    public double R {
      get {
        double xVar = sxCalculator.PopulationVariance;
        double yVar = syCalculator.PopulationVariance;
        var r = covCalculator.Covariance / (Math.Sqrt(xVar) * Math.Sqrt(yVar));
        if (double.IsNaN(r)) return 0.0;
        if (r < -1.0) r = -1.0;
        else if (r > 1.0) r = 1.0;
        return r;
      }
    }

[HeuristicLab-Trac-Bot]

2019-03-16 13:14:39: @gkronber commented

---

r16691: improved code for benchmarking (previous code failed to account for update of trees after call of Optimize as well as early stopping in Optimize for trees with invalid outputs)

[HeuristicLab-Trac-Bot]

2019-03-16 13:25:48: @gkronber commented

---

Benchmark results on my notebook (10 trees, maxSize: 50, Tower)

[[Image(benchmark_16691.png)]]

[HeuristicLab-Trac-Bot]

2019-03-16 13:25:48: @gkronber

[HeuristicLab-Trac-Bot]

2019-03-16 13:26:04: @gkronber uploaded file benchmark_16691.png (27.8 KiB)

[HeuristicLab-Trac-Bot]

2019-04-14 21:16:24: @gkronber changed milestone from HeuristicLab 3.3.16 to HeuristicLab 3.3.17

[HeuristicLab-Trac-Bot]

2019-08-07 13:32:23: @mkommend commented

---

r17193: Merged trunk changes into branch.

[HeuristicLab-Trac-Bot]

2020-01-03 18:00:14: @gkronber commented

---

r17392: merged r17344 to r17351 from trunk to branch

[HeuristicLab-Trac-Bot]

2020-01-03 18:59:56: @gkronber commented

---

r17393: fix typos in comments