search

MansMeg / bayesbenchr

A bayesbench R package for quick posterior analysis
0 stars 0 forks source link

How to use custom inference methods #1

Open eerolinna opened 4 years ago

eerolinna commented 4 years ago

Lets say I have a new inference method implemented in the file mymethod.R

my_inference_method <- function(cfg){
...
}

and I want to run it on 8 schools.

If I use stan_vb I can have a config file like this, right?

- inference_engine: stan_vb
  posterior_name: "8_schools|noncentered"
  ...

Then with my_inference_method can I just have a config file like this?

- inference_engine: my_inference_method
  posterior_name: "8_schools|noncentered"
  ...

How does bayesbench know where to find the R file mymethod.R?

MansMeg commented 4 years ago

Hmm. I do not remember right now. But if I remember correctly it should work if we points to the R file or read it in in the global environment. Need to check though.

eerolinna commented 4 years ago

The python version of bayesbench currently has 

- inference_engine: mymethod.my_inference_method
  posterior_name: "8_schools|noncentered"
  ...

This tells bayesbench where to find the custom inference method. The dot notation is python specific though, so we could have something like

- inference_engine_module: mymethod
  inference_engine: my_inference_method
  posterior_name: "8_schools|noncentered"
  ...

Here inference_engine_module can be anything that python can find, so a local file or a package installed from the package manager. If import mymethod works in python code then bayesbench will find it no matter where it is. I guess library(mymethod) would be the equivalent R code

We could also have inference_engine_file instead of inference_engine_module where the value would be the full path to the file. This works well for local files but not so well for installed packages.

Do you think something like this would be good for the R version too?

eerolinna commented 4 years ago

Any comments?

Also to expand a bit, lets say someone has written R package bayesbench_vi_methods that includes a new VI method called very_nice_vi. I want to use it with bayesbench, I guess the config would need to be something like 

- inference_engine_module: bayesbench_vi_methods
  inference_engine: very_nice_vi
  posterior_name: eight_schools_noncentered