Instances not right in bestalgs' .info files

[brockho]

The instances, displayed in the .info files of the provided reference algorithms ("best algorithms") are potentially wrong. Most likely, only the instances of one (the first?) algorithm used to create the best algorithm are displayed, although the instances actually used might change from function/dimension pair to function/dimension pair. For example, only

1,2,3,4,5,6,7,8,9,10,11,12,13,14,15

are displayed as instances for the (artificial) best algorithm from BBOB-2013 although some of the algorithms have been run on instances 31+.

We shall still do the release 2.0, I suggest, and fix this issue ASAP afterwards.

[dtusar]

You would like to have a list of all instances that were used at least once, right? Is it okay if each instance is shown only once? Because if you generate the best algorithm from 2009 algorithms then you would get a list like this: 1,2,3,4,5,1,2,3,4,5,1,2,3,4,5.

[brockho]

The optimal information will be to know all used instances for each function/dimension/target combination separately. Unfortunately, the current multiobjective data format does simply not provide such per problem information :-( Hence, yes, maybe the first good step is to show all used instances, a reference algorithm is made up from, globally (i.e. over all functions/dimensions/targets). In the long run at least, we should also display how often data for a single instance is used (e.g. like 1(3x),2(3x), ..., 5(3x), 11, 12, ..., 15 or 1: 3,2: 3, ..., 5: 3, 11: 1, 12: 1, ..., 15: 1).

[brockho]

Conclusion: repetitions of instances do not need to be mentioned as this is equivalent to doing restarts.

[dtusar]

Now all different instance lists are used if more than one exists. Each list contains each instance only once (no repetitions). @brockho please check if the result is correct when you'll regenerate the best algorithm.

[dtusar]

This is now solved.