Current sources of variation between BOAT and pyro-boat:
different acquisition function (LCB vs EI)
different inference (SMC/MCMC vs SVI)
different initial points for seeding the optimization
different local optimization packages for optimizing surrogate
different model (BOAT uses two different GPs for the separate residuals)
Performance differences:
For 50 points, both take (wall time) 300s, so about the same
BOAT re-uses cholesky decomposition, Pyro GP does not
BOAT takes longer and longer per step, Pyro version takes the same amount of time always
This somewhat suggests that the GP is not the bottleneck, and instead something else is causing the delay over time with BOAT. Maybe this is due to inference, maybe due to the way the surrogate is optimized.
26/09/2019
Current sources of variation between BOAT and pyro-boat:
Performance differences:
This somewhat suggests that the GP is not the bottleneck, and instead something else is causing the delay over time with BOAT. Maybe this is due to inference, maybe due to the way the surrogate is optimized.