Get BAD working!

[drbenvincent]

The last big big task is to get out Bayesian Adaptive Design working. We already have the experiment and trial loop infrastructure etc. We have models doing inference based on the designs and responses. We have classes for Kirby and Frye et al designs.

• [x] We need a general Abstract Base Class for adaptive designs in general (both DARC and non-DARC) Call it something like AdaptiveABC. Then we can create a concrete class which inherits from both DARCDesign and AdaptiveABC.
• [x] Actually do BAD, by calling optimisation.py. This will involve converting it to work with pandas dataframes.
• [x] Confirm it's working with some parameter recovery simulation, see #6
• [x] Update all the other models given that there was some change in getting BAD to work

[drbenvincent]

Initial classes laid out and the structure basically works. Although this is not doing BAD yet. It does generate all possible designs. But at the moment we just randomly pick from the design space, and we have no implementation of heuristics to cut down the design space. But a good first start.

Next step is to actually do BAD, by calling optimisation.py. This will require sending some model stuff (predictive_y, θ) into back into the design along with the last response. This will also require some updates to the way how we do that, so that we have a consistent method call across all design types.

[drbenvincent]

Ok. Problem found. I was having problems with the matrix algebra when predictive_y was being called from either log_likelihood or design_optimisation as the sizes of the parameters and data are different and it wasn't playing ball.

The problem is that I implemented log_likelihood incorrectly. That needs to iterate over trials, calculating p_chose_delayed, then calculating the product.

[drbenvincent]

Currently, base on some parameter recovery simulations (#6), this is not working as expected. Try to track down the bug. First hunch is in the implementation of the log_likelihood() method.

[drbenvincent]

Actually, it is working. After adding a more rich design space, and trying things out, it's all working fine as far as I can tell. Still have not implemented the heuristic methods of trimming down the design space on each trial, so that should improve things.

[drbenvincent]

Actually no, there IS a problem with design optimisation. Running the Frye task, we get very good inferences, so I think the inference code is working just fine. But the choice of design in the BAD is very poor. This is leading to broad posteriors, but that's the fault of the design selection