numbbo / coco

Numerical Black-Box Optimization Benchmarking Framework
https://numbbo.github.io/coco
Other
262 stars 87 forks source link

[Bug report] cocopp test coverage with background algorithms #1999

Open nikohansen opened 3 years ago

nikohansen commented 3 years ago
import cocopp

cocopp.genericsettings.background = {None: [cocopp.bbob.get('NELDERDO')]}
cocopp.main('2009/DIRECT 2013/CMAES_Hutter b/2009/bay');

works fine. But when the background algorithm is also a main algorithm it fails when generating the tables:

cocopp.genericsettings.background = {None: [cocopp.bbob.get('NELDERDO')]}
cocopp.main('2009/DIRECT 2013/CMAES_Hutter NELDERDO');
Post-processing (2+)
  Using 3 data sets:
[...]
Generating comparison tables...
[DataSet(NELDERDOERR_doerr on f1 2-D), DataSet(NELDERDOERR_doerr 2 on f1 2-D)]
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-17-dd262ecf3fd1> in <module>
      1 """identify bug"""
      2 cocopp.genericsettings.background = {None: [cocopp.bbob.get('NELDERDO')]}
----> 3 cocopp.main('2009/DIRECT* 2013/CMAES_Hutter NELDERDO')

~/opt/anaconda3/lib/python3.7/site-packages/cocopp-2.3.3.18.86-py3.7.egg/cocopp/rungeneric.py in main(argv)
    402             # and rungenericmany.main() or lower-level functions are called.
    403             genericsettings.foreground_algorithm_list = []
--> 404             dsld = rungenericmany.main(args, outputdir)
    405 
    406         toolsdivers.prepend_to_file(latex_commands_filename,

~/opt/anaconda3/lib/python3.7/site-packages/cocopp-2.3.3.18.86-py3.7.egg/cocopp/rungenericmany.py in main(args, outputdir)
    380                     ([1, 20, 38] if (testbedsettings.current_testbed.name ==
    381                                      testbedsettings.suite_name_bi) else True),
--> 382                     latex_commands_file)
    383         print_done()
    384 

~/opt/anaconda3/lib/python3.7/site-packages/cocopp-2.3.3.18.86-py3.7.egg/cocopp/compall/pptables.py in main(dict_alg, sorted_algs, output_dir, function_targets_line, latex_commands_file)
    331                 txt = ("There is more than a single entry associated with "
    332                        "folder %s on %d-D f%d." % (sorted_algs[n], df[0], df[1]))
--> 333                 raise Exception(txt)
    334 
    335             entry = entries[0]

Exception: There is more than a single entry associated with folder /Users/hansen/.cocopp/data-archives/coco.gforge.inria.fr/data-archive/bbob/2009/NELDERDOERR_doerr_noiseless.tgz on 2-D f1.

This is in particular a problem (yet surmountable) when setting the background algorithms like

cocopp.genericsettings.background = {None: cocopp.bbob.get_extended('2009/*')}
brockho commented 3 years ago

I confirm the above and get the same exception on my Windows machine.