Hyper-Parameter Optimization

[watercrossing]

Lets optimise the hyper parameters of matrix factorisation, as there are quite a few parameters. The experiment series class is a starting point, as this shows a neat way of calling wolfe functions. Problem: it currently outputs everything into log files, so we can't get results.

Todo:

• implement gradient free optimisation
• call wolfe code neatly
• integrate into wolfe utilities
• make it a decorator - something like "OptimiseHyperparameters" or similar, that can be thrown in with other wolf code and does the optimisation automagically.

[andreasvlachos]

Another unit test to implement for Nelder-Mead based on the rosenbock function: http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.minimize.html#scipy.optimize.minimize

DONE.

[andreasvlachos]

An alternative to Nelder-Mead is the Powell method using conjugate directions mentioned here: http://docs.scipy.org/doc/scipy-0.14.0/reference/tutorial/optimize.html

[andreasvlachos]

Scipy implementation of Nelder-Mead to check against;

https://github.com/scipy/scipy/blob/maintenance/0.14.x/scipy/optimize/optimize.py

[riedelcastro]

This is nice! I wonder whether we can take this code and adapt it to work with Terms?

[andreasvlachos]

It is probably feasible! But I'd need to understand what are the Terms first!