search

numbbo / coco

Numerical Black-Box Optimization Benchmarking Framework
https://numbbo.github.io/coco
Other
254 stars 85 forks source link

Missing data file for noisy regression tests #2272

Closed olafmersmann closed 5 days ago

olafmersmann commented 2 months ago

We are missing the file bbob_noisy_regression_data_10.json for the noisy regession tests.

FMGS666 commented 2 months ago

Hello, I guess you are talking about the data from the legacy code right? The regression data for the new suite is already in the numbbo/regression-test repository, both 2 and 10. I haven't uploaded the file containing the data for the regression test against the legacy code (basically the one needed to run this script), as it is pretty big (around 380 MB). I have a script for creating such data from the legacy code here. If you want I can upload the data file to coco but to be honest it didn't seem optimal to me to do so, considering the size of the file. Let me know how you prefer to handle the situation

brockho commented 1 week ago

I guess this can be closed. We have the bbob_noisy_regression_data_10 file in the regression-tests repository and we indeed do not need the file, resulting from the experiment of Random Search, because we have the script to create them via the above link.

brockho commented 5 days ago

We do have to consider #2305 though, i.e. turning off the standard regression tests for bbob-noisy (because they are actually non-reproducible) but instead write independent tests for the noiseless and the noisy part.