search

automl / nas_benchmarks

BSD 3-Clause "New" or "Revised" License
91 stars 25 forks source link

Possible error in the implementation of objective_function #13

Closed manonthegithub closed 5 years ago

manonthegithub commented 5 years ago

I suspect that in this line you should have budget and not the constant 100 in the divisor: https://github.com/automl/nas_benchmarks/blob/50ac6126b54159a6f592a30f777f4ffa1286f0f8/tabular_benchmarks/fcnet_benchmark.py#L54 and here: https://github.com/automl/nas_benchmarks/blob/50ac6126b54159a6f592a30f777f4ffa1286f0f8/tabular_benchmarks/fcnet_benchmark.py#L76 and here: https://github.com/automl/nas_benchmarks/blob/50ac6126b54159a6f592a30f777f4ffa1286f0f8/tabular_benchmarks/fcnet_benchmark.py#L96

manonthegithub commented 5 years ago

Now I think, I am probably wrong. You have 100 because you have pretrained models for 100 epochs?

aaronkl commented 5 years ago

yes that's correct probably one should make that more clear