search

heal-research / HeuristicLab

HeuristicLab - An environment for heuristic and evolutionary optimization
https://dev.heuristiclab.com
GNU General Public License v3.0
39 stars 16 forks source link

Gaussian Process Regression Algorithm #1902

Closed HeuristicLab-Trac-Bot closed 11 years ago

HeuristicLab-Trac-Bot commented 12 years ago

Issue migrated from trac ticket # 1902

milestone: HeuristicLab 3.3.8 | component: Algorithms.DataAnalysis | priority: medium | resolution: done

2012-07-25 11:24:53: @gkronber created the issue

HeuristicLab-Trac-Bot commented 12 years ago

2012-07-25 11:31:49: @gkronber commented

r8323: initial import of Gaussian process regression algorithm

HeuristicLab-Trac-Bot commented 12 years ago

2012-07-25 11:32:05: @gkronber changed status from new to accepted

HeuristicLab-Trac-Bot commented 12 years ago

2012-07-25 12:02:54: @gkronber commented

r8324:

  • fixed build (be ready for the obligatory round of beers at the archery meeting)
  • set copy-local to false for new references
HeuristicLab-Trac-Bot commented 12 years ago

2012-07-25 12:08:30: @gkronber commented

r8325: changed return value type for parameter properties

HeuristicLab-Trac-Bot commented 12 years ago

2012-07-31 11:20:08: @gkronber commented

r8366 added linear mean and covariance function

HeuristicLab-Trac-Bot commented 12 years ago

2012-07-31 11:36:50: @gkronber commented

r8368: fixed cloning of linear mean function

HeuristicLab-Trac-Bot commented 12 years ago

2012-07-31 18:26:10: @gkronber commented

r8371: worked on Gaussian Process algorithm

HeuristicLab-Trac-Bot commented 12 years ago

2012-07-31 21:09:07: @gkronber commented

r8372: removed unnecessary plugin references.

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-01 14:46:11: @gkronber commented

r8375: implemented Gaussian process regression operators and analyzers

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-02 17:27:57: @gkronber commented

r8396: implemented LM-BFGS algorithm and improved GPR

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-02 17:39:52: @gkronber commented

r8397: added random seed parameters for LM-BFGS

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-02 17:53:57: @gkronber commented

r8399: fixed plugin dependencies

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-02 19:37:27: @gkronber commented

r8403: introduced upper limit for scale of SEard covariance function

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-06 15:02:53: @gkronber commented

r8416: worked on sum and product covariance functions and fixed a few bugs.

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-06 16:17:15: @gkronber commented

r8417: added periodic covariance function

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-06 18:53:56: @gkronber commented

r8419: changed initialization of hyperparameter vector for GPR using a PRNG

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-07 09:52:36: @gkronber commented

r8421: fixed default mean and covariance function for GPR

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-09 09:34:00: @gkronber commented

r8455: changed calculation of gradients for covariance functions to reduce allocations of arrays

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-09 16:11:10: @gkronber commented

r8439:

  • implemented unit tests for mean and covariance functions.
  • implemented sum and product mean function.
  • fixed incorrect gradient calculation in CovarianceProd
HeuristicLab-Trac-Bot commented 12 years ago

2012-08-09 16:33:24: @gkronber

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-09 16:33:24: @gkronber commented

r8463: improved GPR implementation:

  • implemented additional unit tests
  • fixed bugs uncovered by unit tests
  • improved performance
  • added views for sum and product mean functions
HeuristicLab-Trac-Bot commented 12 years ago

2012-08-09 17:02:35: @gkronber commented

r8464: added const and noise covariance functions.

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-10 20:44:39: @gkronber commented

r8473

  • added line chart
  • made parameters of mean and covariance functions readable
  • removed target variable scaling
  • moved noise hyperparameter for likelihood function to the end of the parameter list
  • added methods to calculate the predicted variance
  • removed limits for scale of covariance functions and introduced exception handling to catch non-spd or singular cov matrixes
  • implemented rational quadratic covariance function
  • added unit test case from GBML book - however, it does not work as the book seemingly uses a noise-less likelihood function
HeuristicLab-Trac-Bot commented 12 years ago

2012-08-11 14:47:04: @gkronber commented

r8475: fixed bug in calculation of variance in GPR model

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-14 13:25:53: @gkronber commented

r8484:

  • changed interface for covariance functions to improve readability
  • fixed several bugs in the covariance functions
  • fixed bugs in the line chart for Gaussian process models
HeuristicLab-Trac-Bot commented 12 years ago

2012-08-14 15:25:01: @gkronber commented

r8489: removed caches in product and sum covariance functions and fixed bug in SE-ARD covariance function

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-14 17:10:19: @gkronber commented

r8491: fixed test cases, improved performance

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-16 13:57:58: @abeham commented

I tested GPR on my dataset and found that CovarianceProd, CovarianceSum, and CovariancePeriodic throw an exception in the GaussianProcessRegressionSolutionCreator. Is there anything I need to take care of in these covariance functions?

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-16 16:34:38: @gkronber

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-16 16:34:38: @gkronber commented

r8494 to r8495:

  • changed mean and covariance function parameters to simple value parameters instead of constrained value parameters
  • changed the solution creator so that it doesn't throw an exception if no model could be found.
HeuristicLab-Trac-Bot commented 12 years ago

2012-08-16 16:37:05: @abeham commented

I would prefer constrained value parameters. Otherwise you can't use them in the create experiment dialog. What's the motivation for moving to a value parameter?

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-16 16:37:30: @gkronber commented

Replying to [comment:27 abeham]:

I tested GPR on my dataset and found that CovarianceProd, CovarianceSum, and CovariancePeriodic throw an exception in the GaussianProcessRegressionSolutionCreator. Is there anything I need to take care of in these covariance functions?

The sum and product covariance functions need must be configured to have at least one child. The periodic covariance function only works for the 1d case (one input variable). With r8494 I changed the parameters so that it becomes more obvious that the sum and product covariance function need to be configured correctly. I also changed the solution creator so that it doesn't throw an exception.

HeuristicLab-Trac-Bot commented 12 years ago

2012-08-16 16:38:35: @gkronber commented

Sum and product covariance function cannot be used in the create experiment dialog anyway.

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-04 09:51:00: @gkronber commented

r8562: implemented LinearARD and MaternIso covariance functions.

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-04 11:26:13: @gkronber commented

r8565: implemented RQard covariance function.

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-05 16:22:42: @gkronber commented

r8580: fixed coloring in line chart for GPR

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-05 17:04:42: @gkronber commented

r8582:

  • implemented a few covariance functions as parameterized named items.
  • Implemented rudimentary view for Gaussian process models.
HeuristicLab-Trac-Bot commented 12 years ago

2012-09-05 23:06:56: @gkronber commented

r8583: fixed build fail.

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-06 16:15:26: @gkronber commented

r8592: added default ctor for HyperParameter

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-10 13:29:10: @gkronber commented

r8612: implemented all mean and covariance functions with parameters as ParameterizedNamedItems

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-10 13:35:10: @gkronber commented

r8614: removed unused file

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-10 13:42:47: @gkronber commented

r8615: renamed covariance functions

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-10 15:09:47: @gkronber commented

r8620: renamed more files. implemented scale covariance function.

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-10 17:13:57: @gkronber commented

r8623: implemented LS Gaussian Process classification

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-21 18:45:26: @gkronber commented

r8678: added masking covariance function and made necessary changes to interface and utility class.

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-21 18:48:32: @gkronber commented

r8679:

  • changed DiscriminantFunctionClassificationModel and DiscriminantFunctionClassificationSolution to non-abstract.
  • Classification based on GPR uses these classes to create a classification solution from a GaussianProcessRegressionModel.
  • Also added view classes for DiscriminantFunctionClassificationModel
HeuristicLab-Trac-Bot commented 12 years ago

2012-09-23 01:19:01: @abeham commented

I noticed that the solution in the run object is only a result collection, but not a DiscriminantFunctionClassificationSolution anymore.

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-23 12:18:47: @gkronber commented

r8682:

  • fixed bug in cloning constructor of CovarianceMask
  • removed throwing Exceptions in CovariancePeriodic (the behaviour of the periodic covariance function now matches the GPML implementation again.
HeuristicLab-Trac-Bot commented 12 years ago

2012-09-23 12:22:16: @gkronber commented

Replying to [comment:45 abeham]:

I noticed that the solution in the run object is only a result collection, but not a DiscriminantFunctionClassificationSolution anymore.

Thanks for the hint! The cloning method override is missing.

HeuristicLab-Trac-Bot commented 12 years ago

2012-09-23 12:23:59: @gkronber commented

r8683: added missing cloning method override in DiscriminantFunctionClassificationSolution