Multi task Gaussian process regression

[danielkorzekwa]

Background:

Predicting sales across different items/stores using multi task gp model with shared parameters is fairly limited. Gaussian Process Regression Networks should help in modelling correlations between items/stores.

Test data: Walmart umbrella competition.

To read:

Multi output GP - shared covariance function on input-dependent features

• Bonilla et al. Multi-task Gaussian Process Prediction, 2008
• mtgp toolbox, Implements the multi-task model of Bonilla et al. Multi-task Gaussian Process Prediction, 2008

Correlated outputs via a linear combination of GPs (fixed W coefficient matrix for combining multiple GPs)

• Semiparametric Latent Factor Models
• Semiparametric Latent Factor Models - technical report
• Multivariate Nonlinear Regression with Semiparametric Latent Factor Models
• Nguyen et al. Collaborative Multi-output Gaussian Processes, 2014

GPRN - Correlation between multiple outputs are adaptive. Every cell of W matrix for mixing multiple GPs is a Gaussian Process itself.

• Gaussian Process Regression Networks
• Gaussian Process Regression Networks (section: Gaussian Process Regression Networks)
• Efficient Variational Inference for Gaussian Process Regression Networks

Convolved GPs - Convolving kernel functions for multiple outputs with Gaussian white noise allows for obtaining a positive semidefinite covariance matrix

• Boyle et al. Dependent Gaussian Processes, 2005
• Boyle et al. Multiple Output Gaussian Process Regression. Technical Report, 2005
• Alvarez et al. Computationally Efficient Convolved Multiple Output Gaussian Processes, 2011

Variational inference + Stochastic variational inference

• Variational Learning of Inducing Variables in Sparse Gaussian Processes - IMPLEMENTED
• Fast Variational Inference in the Conjugate Exponential Family
• Hoffman et al. Stochastic Variational Inference, 2012
• Hensman et al. Gaussian Processes for Big Data, 2013

Other multi output algorithms for non GP models

• Implementation and Application of the Curds and Whey Algorithm to Regression Problems