search

brandontrabucco / design-bench

Benchmarks for Model-Based Optimization
MIT License
80 stars 19 forks source link

Function value mismatch between dataset and the exact oracle in Ant Morphology task #23

Open yunyuewei opened 3 months ago

yunyuewei commented 3 months ago

Hi, I find that there is a mismatch about function value between collected dataset and the exact oracle given the same input in Ant Morphology task. Here is the test code:

import design_bench as db
task = db.make('AntMorphology-Exact-v0')

for i in range(10):
    x = task.x[i]
    y = task.y[i]
    y_pred = task.predict(x.reshape(1, -1))
    print('check', y_pred, y, y==y_pred)

The output is:

check [[-210.13147]] [-210.12979] [[False]]
check [[-71.94952]] [-71.94952] [[ True]]
check [[-252.6108]] [-236.94649] [[False]]
check [[-249.16559]] [-263.27252] [[False]]
check [[-120.36561]] [-120.36561] [[ True]]
check [[-26.405956]] [-26.413733] [[False]]
check [[-197.94395]] [-202.48026] [[False]]
check [[-215.65082]] [-215.64603] [[False]]
check [[-211.04497]] [-211.04497] [[ True]]
check [[-179.61769]] [-179.61769] [[ True]]
check [[-220.93028]] [-217.5053] [[False]]

I also check the DKitty task, and find the oracle matches the values in the dataset. Could you help me resolve this issue?