copulas via autodiff - Githubissues

From Ben Goodrich on stan-dev:

It is kind of like the ODE thing but going the other way. In the case of a copula, we know the explicit form of the multivariate CDF of the marginal uniforms

F(u_1, u_2, ..., u_d | theta)

but for Stan's purposes we need the (logarithm of the) copula density function, which is

dF(u_1, u_2, ..., u_d | theta)
------------------------------ = f(u_1, u_2, ..., u_d | theta)
du_1,du_2,...,du_d

but for large d, there is sometimes no explicit form of f(u_1, u_2, ..., u_d | theta). So, loosely speaking, it would be great if we could evaluate

var f_log(var_vector u, var theta) {
  const int d = u.rows();
  var p = F(u, theta);
  var dp_du_1 = get_partial(p, u[1]);
  var dp_du_1_du_2 = get_partial(dp_du_1, u[2]);
  /* more of these */
  var log_density = log(get_partial(dp_du_*, u[d]);
  return log_density;
}

I replied:

We can do this with second-order autodiff. But it's not going to be super efficient in high dimensions, because each dimension is going to involve a forward pass on the function F, unless we can figure out how to do nested evaluation the way we are building up the ode solver.

The get_partial() operation you have will be a forward-mode auto-diff d_ value. That can then be logged, or whatever, and autodiffed through for HMC.

If the F doesn't involve any pdf or cdf calls in Stan, could do it now. Otherwise, we're going to have to wait until the 2nd order autodiff is completed. It's held up on Windows testing at the moment.

Are there a fixed number of these, or do people write their own F?

To which Ben replied:

There are potentially an infinite number of copula CDFs but in practice, people can only use the ones that have an explicit copula density function for estimation. But also in practice, they estimate the parameters of each margin separately and then pretend those estimates are fixed when they estimate the copula part. Full Bayesian estimation would be much better if we could make it feasible in high dimensions.

stan-dev / stan

copulas via autodiff #938