search

INSIGNEO / openBF

1D blood flow model
https://INSIGNEO.github.io/openBF
Other
28 stars 16 forks source link

Gaussian Process Regression? #16

Closed ChrisRackauckas closed 6 years ago

ChrisRackauckas commented 6 years ago

I'm curious why you used GPy instead of https://github.com/STOR-i/GaussianProcesses.jl . I was also wondering if you tried a bunch of mean and kernel functions to see whether they made much of a difference. We'll be using some of these ideas to implement a cross-diffeq GP regression method for DifferentialEquations.jl based on yours

alemelis commented 6 years ago

I guess you are referring to my thesis, am I right?

GPy is developed here in Sheffield by the CS machine learning group. They periodically organise seminars and workshops on GP and GPy is very well maintained. The number of kernels already implemented in GPy is more than enough to experiment and the kernel combination mechanism works perfectly. Moreover, being written in Python, I could easily use it along with scikit-learn library which eases a lot the data pre-processing and the performance assessment of the final emulator.

In my study cases I experimented with stationary homogeneous kernels (e.g. squared exponential and Matern) and combinations of periodic and non-stationary functions. Of course, they made a lot of difference, and there is not a single kernel good for all the cases. Regarding the mean function, you can always refactor your data to have a zero mean; in this view, the mean function selection is not that important.

I didn't know about GaussianProcess.jl, thank you for pointing it out. I'll give it a look and possibly run a comparison between the two if this can help.

ChrisRackauckas commented 6 years ago

I guess you are referring to my thesis, am I right?

Yup, I took a look at it and it seems like a good general algorithm to add to DifferentialEquations.jl so I started perusing through your materials on it. Good to know about the kernels: I assumed they were all similar, but I guess that's not the case.

Did you notice some kind of "edge" where GP predictions started to not do so well? For example, I assume they won't do well in the region where there's a bifurcation in parameter space, but I didn't see a discussion on that. I assume there must be a "well-behaved" assumption somewhere?

alemelis commented 6 years ago

Yes, you make the assumption that the interpolating function will be locally "smooth". The homogeneous kernels compute the arguments similarity w.r.t. their distance. Therefore, you do not expect abrupt local changes in the input space. This is not a restrictive one in my case as you expect the vascular system to change slowly while in physiological conditions.

However, you can also experiment with discontinuous kernels. You can use step functions and train the emulator with it. It has been proved to work but it requires a deep knowledge of your input data. This is something not always possible when the input space has multiple dimensions.

If you are interested in implementing the GP algorithm, I recommend you the must-read Rasmussen and Williams book on the topic. Rasmussen CE, Williams CK. Gaussian process for machine learning. MIT press; 2006. Another important author on the topic is O'Hagan (and recently Oakley) of whom you can find the relevant papers referenced in my thesis