Feature/zen update

[mlooz]

This is the structure I want to use for #22 .

• The state is kept by the member variables m_opt, m_wsp, m_par, m_cnt and a shared pointer m_wr to a detail::worhp_raii object.
• The normal evolution call does not touch any of these and remains const.
• To do sensitivity updates, a user must call zen_init first, which runs the same worhp optimization as evolution, but saves the resulting state. Then, zen_update can be called to get new optimal values for changed constraints.
• The shared pointer ensures that WorhpFree is called when the last copy of this worhp instance goes out of scope, it should thus be save under copying, moving, etc. Edit: It doesn't solve all the problems. When zen_init is called on an instance and the instance copied afterwards, the copy will share part of its state with the original, creating some confusion.

While a prototype of the functions is there, I haven't tested them yet. There are a couple of open questions:

• As done in the evolution function, I do the linking to the worhp library at runtime also in the zen_update function, which is supposed to be called often. Is the overhead significant?
• Would it be possible to move the function binding into the constructor, or would this mess up the serialization?
• Is it a problem if worhp_raii is declared in the header?
• The perturbation model in Worhp Zen includes a nonlinear perturbation p as a second argument to the fitness function: f(x,p). Should this be included? If so, how?

@bluescarni , @darioizzo and @gabor-varga: Comments on this are very welcome!

Next steps for me:

• [x] Checks for proper state and arguments in the interface to Worhp
• [x] Potentially refactor runtime library linking
• [x] Make sure state is consistent under copying
• [x] Expose zen update in pygmo
• [x] Test case for zen update
• [x] Test case for serialization (archipelago) in different states
• [x] Fixing all the bugs uncovered by the above tests
• [x] Documentation

Edit: Switching to "Ready for Review" to trigger the CI, but please don't merge it yet.

[gabor-varga]

To do sensitivity updates, a user must call zen_init first, which runs the same worhp optimization as evolution, but saves the resulting state. Then, zen_update can be called to get new optimal values for changed constraints.

Do you mean doing an entire new optimisation? It's fine for now, but it would be enough to verify feasibility and optimality conditions are met (I guess running the optimisation is the simplest way).

The perturbation model in Worhp Zen includes a nonlinear perturbation p as a second argument to the fitness function: f(x,p). Should this be included? If so, how?

I think this can be omitted from the interface for now. It could be added without the interface to provide the partials, in which case they would be computed numerically by WORHP.

[mlooz]

Do you mean doing an entire new optimisation? It's fine for now, but it would be enough to verify feasibility and optimality conditions are met (I guess running the optimisation is the simplest way).

I mean that if a user knows they want to look at the sensitivity afterwards, they would call zen_init instead of evolve. It does the same but saves the state. I would like to keep the original evolve unchanged if possible.

I think this can be omitted from the interface for now. It could be added without the interface to provide the partials, in which case they would be computed numerically by WORHP.

Maybe I misunderstood what you mean, but I think that the perturbations are unrelated to partial derivatives. Instead this particular perturbation p targets a fitness function that accepts additional arguments. Which we can add if needed, no problem, it just moves a bit further away from the pygmo problem.

Extracting the derivatives from the problem is an internal matter that should not affect the interface.

[darioizzo]

@mlooz Note that this solution you are trying to go for is not going to work with pagmo.

It is a solution to wrap in python some worhp functionality independently from pagmo.

[darioizzo]

ALso you are basically rewriting evolve with a different name and adding the state management right? It seems to me this structure is a work around rather than a solution ... :(

[mlooz]

ALso you are basically rewriting evolve with a different name and adding the state management right? It seems to me this structure is a work around rather than a solution ... :(

Exactly. Based on the requirements that 1) the zen_update needs state and 2) the pagmo algorithm class should have as little state as possible, I separated them as much as I could without duplicating code.

If this doesn't fit into pagmo, though, I'll try something else.

[gabor-varga]

ALso you are basically rewriting evolve with a different name and adding the state management right? It seems to me this structure is a work around rather than a solution ... :(

We could also pursue a solution outside of pagmo. The idea was that most of the infrastructure is already there for WORHP (loading binary, binding functions).

[mlooz]

Restructured the code as discussed. As of now, the zen update sometimes works and sometimes segfaults, currently hunting down why.

[mlooz]

I think this is getting close to finished. Two small questions:

• I wrote a custom copy constructor to not copy the workspace data structures, because they hold pointers. Should the other mutable members m_last_opt_res, m_log and the cache be copied (more convenient) or not (more consistent)?
• Pagmo has switched to the readthedocs sphinx theme, while ppnf still has the bootstrap theme. Switch?

[mlooz]

This is now ready for review. I realize that I could also have used the m_bool_opts dict instead of WorhpGetBoolParam, I don't mind either way.