Gaussian variational approximation to random intercept model is very slow

[certifiedwaif]

Very slow to converge relative to the Laplacian approximation

[JohnOrmerod]

Dear Mark, Convergence in terms of number of iterations that GVA needs to run, or speed of each iteration? Kind regards, John

---

From: Mark Greenaway [notifications@github.com] Sent: Monday, May 12, 2014 11:41 AM To: certifiedwaif/phd Subject: [phd] Gaussian variational approximation to random intercept model is very slow (#3)

Very slow to converge relative to the Laplacian approximation

— Reply to this email directly or view it on GitHubhttps://github.com/certifiedwaif/phd/issues/3.

[certifiedwaif]

The speed of each iteration. It takes very few iterations to converge.

Pleasingly, it converged to the right value without any modification.

On Mon, May 12, 2014 at 11:48 AM, JohnOrmerod notifications@github.comwrote:

Dear Mark, Convergence in terms of number of iterations that GVA needs to run, or speed of each iteration? Kind regards, John

---

From: Mark Greenaway [notifications@github.com] Sent: Monday, May 12, 2014 11:41 AM To: certifiedwaif/phd Subject: [phd] Gaussian variational approximation to random intercept model is very slow (#3)

Very slow to converge relative to the Laplacian approximation

— Reply to this email directly or view it on GitHub< https://github.com/certifiedwaif/phd/issues/3>.

— Reply to this email directly or view it on GitHubhttps://github.com/certifiedwaif/phd/issues/3#issuecomment-42790472 .

[JohnOrmerod]

Dear Mark, This is what I would expect. The main thing slowing it all down is the number of parameters used to parameterise Lambda. Using an Cholesky parameterisation of the inverse should speed things up dramatically. John

From: Mark Greenaway [notifications@github.com] Sent: Monday, May 12, 2014 11:53 AM To: certifiedwaif/phd Cc: John Ormerod Subject: Re: [phd] Gaussian variational approximation to random intercept model is very slow (#3)

The speed of each iteration. It takes very few iterations to converge.

Pleasingly, it converged to the right value without any modification.

On Mon, May 12, 2014 at 11:48 AM, JohnOrmerod notifications@github.comwrote:

Dear Mark, Convergence in terms of number of iterations that GVA needs to run, or speed of each iteration? Kind regards, John

---

From: Mark Greenaway [notifications@github.com] Sent: Monday, May 12, 2014 11:41 AM To: certifiedwaif/phd Subject: [phd] Gaussian variational approximation to random intercept model is very slow (#3)

Very slow to converge relative to the Laplacian approximation

— Reply to this email directly or view it on GitHub< https://github.com/certifiedwaif/phd/issues/3>.

— Reply to this email directly or view it on GitHubhttps://github.com/certifiedwaif/phd/issues/3#issuecomment-42790472 .

— Reply to this email directly or view it on GitHubhttps://github.com/certifiedwaif/phd/issues/3#issuecomment-42790608.

[certifiedwaif]

Okay, good. That suggests a way forward then.

I just tried the zero-inflated versions of the tests, and the Laplacian and Gaussian approximations both seem to converge to the right parameter values with the right amount of zero inflation.

So I think we're at the point where we can calculate lower bounds, check accuracy and the like.

On Mon, May 12, 2014 at 11:57 AM, JohnOrmerod notifications@github.comwrote:

Dear Mark, This is what I would expect. The main thing slowing it all down is the number of parameters used to parameterise Lambda. Using an Cholesky parameterisation of the inverse should speed things up dramatically. John

From: Mark Greenaway [notifications@github.com] Sent: Monday, May 12, 2014 11:53 AM To: certifiedwaif/phd Cc: John Ormerod Subject: Re: [phd] Gaussian variational approximation to random intercept model is very slow (#3)

The speed of each iteration. It takes very few iterations to converge.

Pleasingly, it converged to the right value without any modification.

On Mon, May 12, 2014 at 11:48 AM, JohnOrmerod notifications@github.comwrote:

Dear Mark, Convergence in terms of number of iterations that GVA needs to run, or speed of each iteration? Kind regards, John

---

From: Mark Greenaway [notifications@github.com] Sent: Monday, May 12, 2014 11:41 AM To: certifiedwaif/phd Subject: [phd] Gaussian variational approximation to random intercept model is very slow (#3)

Very slow to converge relative to the Laplacian approximation

— Reply to this email directly or view it on GitHub< https://github.com/certifiedwaif/phd/issues/3>.

— Reply to this email directly or view it on GitHub< https://github.com/certifiedwaif/phd/issues/3#issuecomment-42790472> .

— Reply to this email directly or view it on GitHub< https://github.com/certifiedwaif/phd/issues/3#issuecomment-42790608>.

— Reply to this email directly or view it on GitHubhttps://github.com/certifiedwaif/phd/issues/3#issuecomment-42790749 .

[certifiedwaif]

http://seattle.intel-research.net/people/jhightower/pubs/fox2003bayesian/fox2003bayesian.pdf

[certifiedwaif]

http://wbnicholson.wordpress.com/2014/07/10/parallelization-in-rcpp-via-openmp/

[certifiedwaif]

http://nbviewer.ipython.org/url/blaze.pydata.org/notebooks/timings-csv.ipynb