Is it possible to use GaussianProcesses to fit an ongoing stream of data?

[schlichtanders]

Hi, I am looking through probabilistic packages in Julia for modeling timeseries in a real-time such that my current model can be efficiently updated given a couple of new data points (maybe even one data point).

With Bayesian Modeling in general, I understood that I can put my previous state simply into the prior and practically use any Bayesian inference algorithm. I would like to understand whether I can do similar things with GaussianProcesses.jl

What I've found so far:

• the default optimization uses maximum likelihood, which to the best of my understanding needs all historic data every time (which is not really an option in my streaming applications)
• I found that BayesianOptimization refers to GaussianProcesses.jl and allows to set priors.

Would the BayesianOptimization the recommended approach for incremental ongoing learning?

[maximerischard]

Hi @schlichtanders. There is some functionality implemented to add one observation at a time. Have a look at GPElastic and see if that could solve your problem. If I remember correctly, this was developed specifically to support this use case in the BayesianOptimization package. I should say this is for the exact case, not for MCMC with non-Gaussian likelihoods.

[schlichtanders]

Thank you very much. Can GPElastic also be used with the standard maximumlikelihood optimizer? Or is it only valid for the BayesianOptimization (e.g. following their intro example)

[schlichtanders]

Another followup: Would this timeseries example with trend and cyclic information work with GPElastic? https://stor-i.github.io/GaussianProcesses.jl/latest/mauna_loa/

[maximerischard]

Yes, it should work with the standard maximum likelihood optimizer. But if I'm not mistaken, when you optimize the (covariance kernel and mean) hyperparameters, the algorithm will need to re-calculate the covariance matrix and its Cholesky decomposition at each step, so you will not see performance benefits from using GPElastic instead of GPE. You should expect to see a performance benefit when generating updated predictions with new observations while retaining the same hyperparameters.

[maximerischard]

Another followup: Would this timeseries example with trend and cyclic information work with GPElastic? https://stor-i.github.io/GaussianProcesses.jl/latest/mauna_loa/

I would expect it to work, yes. Let us know if it doesn't.