Standardizing the interfaces for optimizer::Solver classes

[mxgrey]

Right now the abstract optimizer::Solver class is extremely minimalistic, only offering the ability to solve a Problem that was passed in during construction (incidentally, I think it should be possible to swap out the current Problem with another Problem).

The different implementations of the Solver class have many similar parameters that could easily make use of a common interface. For example: maximum number of iterations, tolerance for solution, specify file to output data to, frequency of printing progress to the terminal, and maybe other things that I'm not thinking of right now.

Right now these parameters are hidden within each different Solver implementation, so I propose making a unified interface in the abstract Solver class. That should make it easier to swap out different Solver implementations.

On an unrelated note, I plan on making a default DART Solver implementation (just a simple stochastic gradient descent algorithm) as an option that doesn't depend on third-party libraries.

[mxgrey]

Also, I'd like to use shared_ptrs for Problems so that they can be safely swapped and shared between solvers without needing to worry about memory management for them.

[jslee02]

All sounds good to me.

Speaking of optimizer: This might not be directly related to this proposal. Do you plan to provide API for linear or quadratic functions as a objective function or constraint functions? Current third-party libraries supported by DART (NLOPT, IPOPT, SNOPT(not implemented yet)) are all solvers for nonlinear programming but we can consider other solvers such as qpOASES (quadratic programming solver).

[mxgrey]

That's an interesting idea. Off the top of my head, I imagine we'd need to separate out the nonlinear Problems, Functions, and Solvers from the Linear/Quadratic Problems, Functions, and Solvers. But it could be worth thinking about how those classes might overlap, and we can come up with a coherent interface for them.

[mxgrey]

I guess Linear/Quadratic Functions are really a specialization of nonlinear Functions, so it might be perfectly reasonable to have a QuadraticFunction class that inherits the Function class, and allows the end user to specify the coefficients for the equation. We'd also offer a QuadraticProblem class that inherits the Problem class but only accepts QuadraticFunctions.

Then you could still pass a QuadraticFunction into a regular Problem and use it as normal. Or you could pass a QuadraticProblem into a nonlinear solver and use it as normal. But we'd also offer a QuadraticSolver that only accepts QuadraticProblems and solves those using QP methods instead of nonlinear methods.

[mxgrey]

As a sidenote, I won't be able to test all of my changes without having access to the third-party libraries (snopt, nlopt, and ipopt). It looks like nlopt and ipopt shouldn't be an issue, but snopt does not appear to be free, even for academic use. Does anyone have access to snopt and would be able/willing to test the changes that I'm making?

[jslee02]

I agree that Linear/Quadratic Functions are a specialization of nonlinear Functions and the relationship in terms of class inheritance.

Does anyone have access to snopt and would be able/willing to test the changes that I'm making?

I have access to snopt and willing to test the changes. I haven't used snopt yet but would like to learn it.

[mxgrey]

Since I can't test anything with snopt myself, I'm going to be making a bunch of guesses with whatever changes I make to it. It'll be interesting to see how well I manage to do, but I'm sorry to say that you'll probably have to do a lot of fixing ;p

[jslee02]

I guess so, and will take it. Go ahead!

[mxgrey]

In the optimizer::Function::eval() functions we use Eigen::Map<const Eigen::VectorXd>& to pass in the current state variable. I propose changing this to const Eigen::VectorXd& because an Eigen::Map<const Eigen::VectorXd>& can be converted to a const Eigen::VectorXd& with no overhead and no loss of functionality (at least as far as I can tell), and I think it would make the API look a bit nicer.

Unfortunately, I don't think the Eigen::Map<Eigen::VectorXd> arguments can be changed for the gradient and Hessian, but oh well.

[mxgrey]

@jslee02 Since the Hessian is a square matrix, is there a particular reason that Eigen::Map<Eigen::VectorXd, Eigen::RowMajor> was chosen to represent it, instead of Eigen::Map<Eigen::MatrixXd, Eigen::RowMajor>? I imagine that templating it with MatrixXd would make indexing a lot easier for the end user.

I can't find any examples of it being used in the third-party solvers, so I can't really guess at what the motivation would be.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.