search

numbbo / coco

Numerical Black-Box Optimization Benchmarking Framework
https://numbbo.github.io/coco
Other
263 stars 91 forks source link

Postprocessing ignores instance repetitions #2296

Closed nikohansen closed 4 months ago

nikohansen commented 5 months ago

Repetitions that come from the same folder/info-file trigger this warning

cocopp\pproc.py:2384: UserWarning: same DataSet found twice, skipping the second one from ['exdata\\debug-0002\\bbobexp_f1.info'] with instances [1, 2, 3, 4, 5, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80]
  warnings.warn("same DataSet found twice,"

and are hence ignored, which, at this point in time, should be considered as a bug.

nikohansen commented 5 months ago

This should only happen if the data are actually the same (compared in the method DataSet._data_differ), not only the instances.

I am not sure what the right action should be in this case.

ttusar commented 5 months ago

It is the same data.

nikohansen commented 5 months ago

The upcoming ExperimentRepeater used in the upcoming example experiment starts by default from allzeros in 1 of 15 repetitions, which leads with deterministic algorithms to a repeated data set which is then (currently) ignored by the postprocessing.

nikohansen commented 4 months ago

Should be more or less fixed by now?