paul-buerkner / brms

brms R package for Bayesian generalized multivariate non-linear multilevel models using Stan
https://paul-buerkner.github.io/brms/
GNU General Public License v2.0
1.27k stars 182 forks source link

Add more Gaussian process kernels #234

Closed paul-buerkner closed 1 week ago

paul-buerkner commented 7 years ago

Right now, only the exponentiated-quadratic kernel is supported as it has a native implementation in Stan. However, there seem to be quite a few other kernels worth considering. This issue is ment to provide a list of kernels with results concerning their feasibility in Stan.

jae0 commented 6 years ago

Something like the following would cover the main forms discussed in :

[https://en.wikipedia.org/wiki/Mat%C3%A9rn_covariance_function]


functions {
            matrix matern_covariance( int N, matrix dist, real phi, real sigma_sq, int COVFN) {
              matrix[N,N] S;
              real dist_phi; 
              real sqrt3;
              real sqrt5;
              sqrt3=sqrt(3.0);
              sqrt5=sqrt(5.0);

              if (COVFN==1) { // exponential == Matern nu=1/2 , (p=0; nu=p+1/2)
                for(i in 1:(N-1)){
                  for(j in (i+1):N){
                    dist_phi = fabs(dist[i,j])/phi;
                    S[i,j] = sigma_sq * exp(- dist_phi ); 
                }}

              } else if (COVFN==2) { // Matern nu= 3/2 covariance
                for(i in 1:(N-1)){
                  for(j in (i+1):N){
                   dist_phi = fabs(dist[i,j])/phi;
                   S[i,j] = sigma_sq * (1 + sqrt3 * dist_phi) * exp(-sqrt3 * dist_phi);
                }}

              } else if (COVFN==3) { // Matern nu=5/2 covariance
                for(i in 1:(N-1)){
                  for(j in (i+1):N){
                    dist_phi = fabs(dist[i,j])/phi;
                    S[i,j] = sigma_sq * (1 + sqrt5 *dist_phi + 5* pow(dist_phi,2)/3) * exp(-sqrt5 *dist_phi);
                }}

              } else if (COVFN==4) { // Matern as nu->Inf become Gaussian (aka squared exponential cov)
                for(i in 1:(N-1)){
                  for(j in (i+1):N){
                    dist_phi = fabs(dist[i,j])/phi;
                    S[i,j] = sigma_sq * exp( -pow(dist_phi,2)/2 ) ;
                }}
              } 

              for(i in 1:(N-1)){
              for(j in (i+1):N){
                S[j,i] = S[i,j];  // fill upper triangle
              }}

              // create diagonal: nugget(nonspatial) + spatial variance +  eps ensures positive definiteness
              for(i in 1:N) S[i,i] = sigma_sq ;            
              return(S)   ;
            }

      }
paul-buerkner commented 6 years ago

Thanks, this is very helpful!

gabriuma commented 5 years ago

To use the Mattern covfun in approximate GP, we just to change the spectral density function (function 'spd' in the Stan code):

   #Spectral density for Mattern nu=3/2
real spd(real alpha, real rho, real w) {
    real S;
    S = 4*alpha^2 * (sqrt(3)/rho)^3 * 1/((sqrt(3)/rho)^2 + w^2)^2;
    return S;
}

    #Spectral density for Mattern nu=1/2
real spd(real alpha, real rho, real w) {
    real S;
    S = 2*alpha^2 * (1/rho) * 1/((1/rho)^2 + w^2);
    return S;
}

    #Spectral density for Mattern nu=5/2
real spd(real alpha, real rho, real w) {
    real S;
    S = 16/3*alpha^2 * (sqrt(5)/rho)^5 * 1/((sqrt(5)/rho)^2 + w^2)^3;
    return S;
}

All the rest Stan code remains the same.

benearnthof commented 4 years ago

Has there been an update on this? I'm new to this package and would like to use a Matern kernel in a model, but I'm not sure where to put the Stan code snippets.

paul-buerkner commented 4 years ago

I think(?) that matern kernels are not supported in Stan itself so supporting them in brms should not be a major issue. Perhaps I find the time in the upcoming weeks.

benearnthof commented 4 years ago

For future reference: It seems like the Matern covariance functions are available in The Stan Math library

jgradym commented 3 years ago

Just wanted to voice support for adding a Ornstein–Uhlenbeck option for running a brm() function with a phylogeny. This seems to be the new standard evolutionary covariance structure when dealing with phylogenies, as its more realistic than others (ie, consistent with stabilizing selection). An option of model = "OU", "Brownian" etc would be super helpful. As is, I can't use brms for most phylo analyses. Most recent evol packages (using MLE instead of Bayesian) have about 5 standard options (eg see ?phylopars in Rphylopars). But the most important, by far is the OU one. Would be fantastic to get it into brms!

potash commented 3 months ago

Hello, I would like to see the Matern covariance functions (specifically gp_exponential_cov) in brms. Can I help make this happen? I have been browsing the source code and it seems fairly straightforward. Some questions I have:

0) Do you have any documentation to help new contributors? I have never developed an R package. How do I install it from local source? Do I need to recompile/reinstall every time I modify the source?

1) will the brms gp(x, sdgp, lscale, zgp) function in stan take cov as a parameter? Does stan have string parameters or will there be a set of constants int GP_EXP_QUAD_COV=1; int GP_EXPONENTIAL_COV=2; etc.?

2) How can .predictor_gp() get passed the cov parameter?

3) Is it necessary to implement the approximate gp for these covariances as well? I feel this should be done eventually but as I and many others do not necessarily need it, it would be OK and faster to start with only exact versions of these kernels and throw an error for approximate.

Thanks!

potash commented 3 months ago

Edit: I now see @gabriuma's comment about approximate Matern covariances and see that it should be quite easy so perhaps my third question is moot.

paul-buerkner commented 3 months ago

I will add this issue to the list of issues for the next release to make sure I will at least implement the Matern kernels. I will have to think about the best way to implement different kernels in the Stan code so it is a bit hard for me to suggest concrete steps that you could already take to implement this.

paul-buerkner commented 1 week ago

This has been implemented in #1688. I added matern and exponential kernels including HSGP versions of the matern kernels. If people need other kernels at some point, I suggest to open new issues for these specific kernels.

potash commented 6 days ago

Fantastic, thank you! I will install and test it soon, but just a quick note/question: the exponential kernel is a matern kernel (smoothness 1/2), so I was hoping an HSGP approximation is supported but from the way your last comment is worded it seems maybe not?

paul-buerkner commented 6 days ago

Good point. :-D

I didn't see the math of the HSGP for Matern 1/2 in our HSGP paper and I didn't have time to do it myself. If you have time to compute the spectral density for this kernel (if it exists) then it would be easy to implement it.

@avehtari was there a reason we didn't discuss the Matern 1/2 kernel in our paper?

gabriuma commented 6 days ago

Hi,

The spectral density for Matern 1/2 is

s_{\frac{1}{2}}(\bm{\omega})&= \alpha\,\frac{2^D\pi^{D/2}\Gamma(\frac{D+1}{2})}{\sqrt{\pi}\ell}\left(\frac{1}{\ell^2}+\bm{\omega}^\intercal \bm{\omega} \right)^{-\frac{D+1}{2}} \label{eq_specdens_12}

For instance, with input dimensionality $D=3$ and $\bm{\omega}=(\omega_1,\omega_2,\omega_3)^\intercal$, the spectral density takes the form

s{\frac{1}{2}}(\bm{\omega})&= \alpha \, 8\,\pi \prod{i=1}^{3}\elli\left(1+\sum{i=1}^{3}\ell_i^2 \omega_i^2 \right)^{-2}

Hopely it helps.

Best,

Gabriel FISABIO, Environment and Health

El lun, 23 sept 2024 a las 16:02, Paul-Christian Bürkner (< @.***>) escribió:

Good point. :-D

I didn't see the math of the HSGP for Matern 1/2 in our HSGP paper and I didn't have time to do it myself. If you have time to compute the spectral density for this kernel (if it exists) then it would be easy to implement it.

@avehtari https://github.com/avehtari was there a reason we didn't discuss the Matern 1/2 kernel in our paper?

— Reply to this email directly, view it on GitHub https://github.com/paul-buerkner/brms/issues/234#issuecomment-2368365131, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHEXZGTFMIE4IRWGXT6QKF3ZYANPXAVCNFSM6AAAAABJGWVFR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRYGM3DKMJTGE . You are receiving this because you were mentioned.Message ID: @.***>

paul-buerkner commented 6 days ago

Perfect! Thank you!

avehtari commented 6 days ago

@avehtari was there a reason we didn't discuss the Matern 1/2 kernel in our paper?

Trying to get the paper out in finite time?

paul-buerkner commented 6 days ago

Fair point. :-D Thanks!

gabriuma commented 6 days ago

Trying to get the paper out in finite time? basically:-)

We empirically calculated the relationship between HSGP factors (section 4.3) for Matern \inf, 3/2, 5/2, which requires quite a bit of work. In my experience, these may be the most useful kernels for common applications. Matern 1/2 is too rough (wiggly) I think.

Gabriel FISABIO, Environment and Health

El lun, 23 sept 2024 a las 16:53, Paul-Christian Bürkner (< @.***>) escribió:

Fair point. :-D Thanks!

— Reply to this email directly, view it on GitHub https://github.com/paul-buerkner/brms/issues/234#issuecomment-2368528196, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHEXZGVPNOEWKZ2YYFA25LDZYATNLAVCNFSM6AAAAABJGWVFR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRYGUZDQMJZGY . You are receiving this because you were mentioned.Message ID: @.***>

potash commented 6 days ago

@gabriuma Matern 1/2 is fairly common in soils and agriculture, maybe it's just a historical accident that should be corrected... or maybe it makes sense because these variables can be very rough with a lot of variation (not measurement error) on very short length scales. Maybe they would be better represented as a sum of two smoother kernels but matern 1/2 could be a decent choice for simplicity.

paul-buerkner commented 5 days ago

HSGPs for the exponential kernel are now supported (via https://github.com/paul-buerkner/brms/pull/1688)

avehtari commented 1 day ago

How about HSGP with periodic kernel? I was going to test the Birthdays case study with brms + Pathfinder, but would need periodic HSGP

paul-buerkner commented 1 day ago

yeah should be doable. periodic kernel needs a bit more work because it has an additional parameter I think. would you mind opening a new issue for it?

Aki Vehtari @.***> schrieb am Sa., 28. Sept. 2024, 20:37:

How about HSGP with periodic kernel? I was going to test the Birthdays case study with brms + Pathfinder, but would need periodic HSGP

— Reply to this email directly, view it on GitHub https://github.com/paul-buerkner/brms/issues/234#issuecomment-2380843889, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADCW2AADKBMV6EYBWHXGUNTZY3SODAVCNFSM6AAAAABJGWVFR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBQHA2DGOBYHE . You are receiving this because you modified the open/close state.Message ID: @.***>

gabriuma commented 1 day ago

Hi,

We implemented an approximate linear representation of a periodic squared exponential kernel in our paper. Please read section 3.3, then appendix B, then section 4.4, and finally case study 5.2.

Gabriel FISABIO, Environment and Health

El sáb, 28 sept 2024 a las 20:39, Paul-Christian Bürkner (< @.***>) escribió:

yeah should be doable. periodic kernel needs a bit more work because it has an additional parameter I think. would you mind opening a new issue for it?

Aki Vehtari @.***> schrieb am Sa., 28. Sept. 2024, 20:37:

How about HSGP with periodic kernel? I was going to test the Birthdays case study with brms + Pathfinder, but would need periodic HSGP

— Reply to this email directly, view it on GitHub < https://github.com/paul-buerkner/brms/issues/234#issuecomment-2380843889>,

or unsubscribe < https://github.com/notifications/unsubscribe-auth/ADCW2AADKBMV6EYBWHXGUNTZY3SODAVCNFSM6AAAAABJGWVFR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBQHA2DGOBYHE>

. You are receiving this because you modified the open/close state.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/paul-buerkner/brms/issues/234#issuecomment-2380860510, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHEXZGTAVPMFJNAABYZE5G3ZY3ZVHAVCNFSM6AAAAABJGWVFR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBQHA3DANJRGA . You are receiving this because you were mentioned.Message ID: @.***>