search

deephyper / benchmark

Repository to benchmark DeepHyper on different systems.
BSD 2-Clause "Simplified" License
5 stars 3 forks source link

Create Multi-Objective Black-Box Optimization Benchmarks #5

Closed Deathn0t closed 1 year ago

Deathn0t commented 1 year ago

Similarly to C-BBO benchmarks some continuous benchmarks for continuous black-box optimization can should be created. They could be added in the same folder as C-BBO with some specific prefix/suffix to show that they are Multi-Objectives. Then the C-BBO table can be updated accordingly.

Deathn0t commented 1 year ago

Merged https://github.com/deephyper/benchmark/pull/8 which adds the DTLZBenchmark for MOO.

Deathn0t commented 1 year ago

Are JAHS-Bench problems working or do we still need to work on it?

thchang commented 1 year ago

FYI, I think this probably belongs under #6 and we should close this issue, but to answer your question:

Yes I JAHS-Bench working in DeepHyper in my fork:

https://github.com/thchang/benchmark/tree/main/lib/JAHSBench

Need to make some small modifications to let user set problem params from CL and download datasets on install, but otherwise it is working. Note that JAHS-Bench doesn't work on MacOS, so that is one limitation without any workaround.