search

ttusar / coco-gbea

An instance of the COCO platform that contains the GBEA suites
Other
3 stars 0 forks source link

get no html output for rw-top-trumps #1

Closed dietmarwo closed 3 years ago

dietmarwo commented 3 years ago

Running on Mint 20 / Ubuntu Linux, anaconda python 3.8, there is no issue, for coco-gbea/code-experiments/build/python/exdata/random_search_on_bbob_budget0002xD I can successfully get output using

python -m cocopp coco-gbea/code-experiments/build/python/exdata/random_search_on_bbob_budget0002xD

But for

python -m cocopp coco-gbea/code-experiments/build/python/exdata/rw-top-trumps

I get two errors:

There is no Python process running which could block the info file. The files itself look unsuspicious.

ttusar commented 3 years ago

Thanks for letting us know about this issue, I will have a look tomorrow.

ttusar commented 3 years ago

When you write pip freeze in the terminal, which versions of cocoex and cocopp do you have?

ttusar commented 3 years ago

My best guess here is that an incorrect cocopp version is being used (the one from the original COCO repository instead of the one from this repository). I would advise to uninstall any cocopp modules (repeat pip uninstall cocopp several times if needed) and then reinstall it from here again using

python do.py install-postprocessing

BTW, if you pull from this repository now, the version number will be fixed (it was 0.0.0 before and will be 1.0 now).

dietmarwo commented 3 years ago

pip uninstall cocopp + python do.py install-postprocessing fixed the issue, you may close it.

Unfortunately this revealed that there is only small progress related to the random solver. Usually for hard problems I am applying a few thousand optimization retries - paralellized by my optimizer implementation. There are several projects supporting parallel optimization, for instance https://esa.github.io/pygmo2/ and https://github.com/dietmarwo/fast-cma-es . Exploiting paralellization you can compute quite a number of function evaluations.

The shown generated "ERT in number of function evaluations" table shows mostly failures, probably because either the configured goals are not adequate or the number of evaluations is too low. This makes it difficult to measure progress in relation to the random solver.

To apply a parallel optimization algorithm the gbea socket server should enable parallel fitness function execution. Since this is a bit tricky to implement - specially for different programming languages - the "poor mans" solution would be to combine the output of multiple runs to a single one considering always the best solution found. This way the number of function evaluations for the ERT tables could be increased, and if the number of evaluations per run is also increased we would hopefully see more successes in the ERT table to compare with.

ttusar commented 3 years ago

Parallelizing solving of a single problem is not supported, but you can parallelize the solving of several problems (or problem instances).