luinardi / hypermapper

Black-box Optimizer based on Bayesian Optimization
MIT License
155 stars 26 forks source link

Iteration and best point parameter values exceed bounds #50

Closed noahsherrill closed 3 years ago

noahsherrill commented 3 years ago

When running a simple test case in HyperMapper Version 2.2.3, HyperMapper selects parameter values outside of the bounds specified by the "values" item in the configuration file. The configuration file is as follows:

{
  "application_name": "param_values_test",
  "optimization_objectives": ["value"],
  "optimization_iterations": 5,
  "input_parameters": {
    "param_1": {
      "parameter_type": "integer",
      "values": [1, 10],
      "parameter_default": 1
    },
    "param_2": {
      "parameter_type": "integer",
      "values": [10, 20],
      "parameter_default": 20
    }
  }
}

The code is as follows:

from hypermapper import optimizer

def test_function(parameters):
    return (15 - parameters["param_1"] - parameters["param_2"]) ** 2

if __name__ == "__main__":
    parameters_file_path = "param_2_values_test.json"
    optimizer.optimize(parameters_file_path, test_function)

The output is as follows:

Design of experiment phase, number of new doe samples = 10 .......
param_1,param_2,value,Timestamp
1,20,36,2
9,4,4,2
6,7,4,2
4,6,25,2
2,6,49,2
5,7,9,2
4,7,16,2
7,6,4,2
1,6,64,3
7,0,64,3

End of doe/resume phase, the number of evaluated configurations is: 10

Starting optimization iteration 1
param_1,param_2,value,Timestamp
8,4,9,375

Starting optimization iteration 2
param_1,param_2,value,Timestamp
7,4,16,730

Starting optimization iteration 3
param_1,param_2,value,Timestamp
7,10,4,1085

Starting optimization iteration 4
param_1,param_2,value,Timestamp
8,6,1,1467

Starting optimization iteration 5
param_1,param_2,value,Timestamp
7,12,16,1864

End of Bayesian Optimization
Best point found:
param_1,param_2,value
8,6,1

End of HyperMapper

For several of the iterations in the design of experiment phase and in the optimization phase, the value selected for param_2 is outside of the required range [10, 20]. I'm not sure whether or not this is intended behavior. The output reports a best point at the end that also uses an out-of-bounds value for param_2. This does not seem correct.

arturluis commented 3 years ago

Hey @noahsherrill,

We have just release HyperMapper v2.2.4 which fixes this issue. Can you update to the latest version and see if the issue persists?

noahsherrill commented 3 years ago

@arturluis, I no longer see this issue in HyperMapper Version 2.2.4; the parameter values are all in the expected range. This would have been a difficult issue to work around, so I really appreciate the fix!

Also, thank you for making HyperMapper available. We are excited to continue experimenting with it.