numbbo / coco

Numerical Black-Box Optimization Benchmarking Framework
https://numbbo.github.io/coco
Other
260 stars 86 forks source link

Switching the processed suite crashes postprocessing #1500

Closed brockho closed 6 years ago

brockho commented 7 years ago

When postprocessing algorithms from different test suites (e.g. first some bbob algorithms and then some bbob-biobj algorithms) crashes the postprocessing (usage of cocopp.main from within the ipython shell:

ECDF graphs per noise group...
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-6-f5d4eac7ced9> in <module>()
----> 1 cocopp.main("./DEMO.tgz ./GA-MULTIOBJ-NSGA-II.tgz")

C:\Users\dimo\AppData\Roaming\Python\Python27\site-packages\cocopp-2.0.510-py2.7.egg\cocopp\rungener
ic.pyc in main(argv)
    262
    263         if len(args) >= 2 or len(genericsettings.background) > 0:
--> 264             dsld = rungenericmany.main(genopts + ["-o", outputdir] + args)
    265             toolsdivers.prepend_to_file(latex_commands_filename,
    266                                         ['\\providecommand{\\numofalgs}{2+}']

C:\Users\dimo\AppData\Roaming\Python\Python27\site-packages\cocopp-2.0.510-py2.7.egg\cocopp\rungener
icmany.pyc in main(argv)
    462                                 dictAlg[sortedAlgs[0]].getFuncGroups(),
    463                                 inset,
--> 464                                 genericsettings.many_algorithm_file_name)
    465             print_done()
    466

C:\Users\dimo\AppData\Roaming\Python\Python27\site-packages\cocopp-2.0.510-py2.7.egg\cocopp\rungener
icmany.pyc in grouped_ecdf_graphs(alg_dict, order, output_dir, function_groups, settings, parent_fil
e_name)
     61                            outputdir=output_dir,
     62                            info=('%02dD_%s' % (d, gr)),
---> 63                            settings=settings
     64                            )
     65

C:\Users\dimo\AppData\Roaming\Python\Python27\site-packages\cocopp-2.0.510-py2.7.egg\cocopp\compall\
pprldmany.pyc in main(dictAlg, order, outputdir, info, dimension, parentHtmlFileName, plotType, sett
ings)
    670                     displaybest = False
    671                 else:
--> 672                     refalgentry = refalgentries[(dim, f)]
    673                     refalgevals = refalgentry.detEvals(target_values((f, dim)))
    674                     # print(refalgevals)

KeyError: (2, 25)

That's probably not an overly common usecase, but we might still want to have the functionality in the longer run to do that.

brockho commented 7 years ago

After looking at the issue again today, we think, we found the cause: when the bestalg is loaded, it is decided whether it has to be read in from disk or not but there is no check whether the already loaded algorithm is the requested one.

dtusar commented 6 years ago

This is fixed now.