search

numbbo / coco

Numerical Black-Box Optimization Benchmarking Framework
https://numbbo.github.io/coco
Other
262 stars 87 forks source link

Storing the final solution #1995

Open nikohansen opened 3 years ago

nikohansen commented 3 years ago

It could be useful to always record the "final" solution (not evaluation).

Open questions:

brockho commented 3 years ago

Wouldn't the final solution be the last evaluated before the free()?

About the question where to store it: that should be the observer/logger who should know. If there is no observer/logger, there is probably also no need to store it, right? At least it won't be immediately clear then where to write and who should write.

nikohansen commented 3 years ago

Wouldn't the final solution be the last evaluated before the free()?

Or the best-ever solution? Or...

About the question where to store it: that should be the observer/logger who should know. If there is no observer/logger, there is probably also no need to store it, right? At least it won't be immediately clear then where to write and who should write.

The logger should know, indeed, still we have to code the logger and make the decision where it shall go, like in a new file or in the info file or in a data file...

brockho commented 3 years ago

To write into the .info file might be problematic in terms of compatibility. Writing into the .dat files seems okay, the easiest in terms of not breaking anything is of course a separate file (not sure that .fdat would be the right name, maybe .bdat for best data?).

Regarding the final solution, you are right, maybe the best ever is even better. If we decide to write anyway into a separate file, we should probably report both.

ttusar commented 3 years ago

After the last evaluation, we already write something to the .tdat file. It is not the last evaluated solution, but the best one found so far (or in the bi-objective case the best indicator value).

nikohansen commented 3 years ago

Interesting, independent of dimension? Does that mean the .tdat file has inconsistently formatted rows?

ttusar commented 3 years ago

To clarify, we write some information on the best solution - whether it includes its variables or not depends on the problem dimension.

nikohansen commented 3 years ago

Using the .info file as storing medium is rejected by @ttusar and @nikohansen , as it would mess up its nice, clean format and break backwards compatibility, unless we would not just append new summary information but insert it at the right place (which seems messy to do).

nikohansen commented 3 years ago

After discussing @meet. One idea to go about this: we create two files in json format to write meta-data (which are currently not needed for the post-processing). One is written before the run starts, another is written right after the run finished with "meta"-data that depend on the run. In this latter file, the final or best or both solution should be stored.

nikohansen commented 5 months ago

Bringing this back to our attention because the logger is currently under "active" development. Any quick and dirty solution is here probably better than nothing?