timing experiment under changing time complexity

[nikohansen]

the current timing experiment does not well reflect algorithms which have a changing (in particular growing) time complexity. We should try to address this without to introduce otherwise unwarranted and unreasonable experimental complexity.

[brockho]

Shouldn't it be relatively easy to integrate such a timing experiment now that we have the example_experiment_non_anytime.py?

[nikohansen]

True, the cocoex.utilities.ProblemNonAnytime class could indeed make this timing, AFAICS. The only downside is that it is only available in Python.

Pushing this idea further, at least in Python the Problem class could automatically time the experiment and store some related information which could be queried if desired.

[brockho]

Okay. We should keep this in mind when porting the anytime example experiment to the other languages, see #1594.

[nikohansen]

Thinking about it, isn't that precisely the case where we would want a tiny little logger which can be wrapped around a problem doing the job? Should only be very few lines of code, but fully seperated from any Problem class. EDIT: this may be difficult or impossible to do unless the Problem has callback functionality.

[brockho]

A note from today's call: If we want to have (back) a dedicated timing experiment on a single function, we will need that the suite itself defines on which function the experiment should run (not too easy function to be able to record long enough, not too costly function to not disguise the internal runtime of the algorithm).