search

automl / fanova

Functional ANOVA
122 stars 45 forks source link

support for hyperparameters of type Constant #77

Closed shukon closed 5 years ago

shukon commented 5 years ago 
  File "/home/shuki/Documents/ParameterImportance/pimp/importance/importance.py", line 499, in evaluate_scenario
    self.evaluator = method
  File "/home/shuki/Documents/ParameterImportance/pimp/importance/importance.py", line 372, in evaluator
    verbose=self.verbose)
  File "/home/shuki/Documents/ParameterImportance/pimp/evaluator/fanova.py", line 63, in __init__
    seed=self.rng.randint(2**31-1), cutoffs=cutoffs)
  File "/home/shuki/Documents/fanova/fanova/fanova.py", line 79, in __init__
    if (np.max(X[:, i]) > self.cs_params[i].upper) or \
AttributeError: 'ConfigSpace.hyperparameters.Constant' object has no attribute 'upper'
AndreBiedenkapp commented 5 years ago

@shukon could you please check if that is fixed with 541d5a23c75bed7df7fe1f370047c8d1eaaf0593

shukon commented 5 years ago

Thank you for the quick reaction! Fixed that error, but creates related:

  File "/home/shuki/Documents/ParameterImportance/pimp/importance/importance.py", line 499, in evaluate_scenario
    self.evaluator = method
  File "/home/shuki/Documents/ParameterImportance/pimp/importance/importance.py", line 372, in evaluator
    verbose=self.verbose)
  File "/home/shuki/Documents/ParameterImportance/pimp/evaluator/fanova.py", line 63, in __init__
    seed=self.rng.randint(2**31-1), cutoffs=cutoffs)
  File "/home/shuki/ve/cave/lib/python3.6/site-packages/fanova/fanova.py", line 85, in __init__
    if len(unique_vals) > len(self.cs_params[i].choices):
AttributeError: 'ConfigSpace.hyperparameters.Constant' object has no attribute 'choices'
shukon commented 5 years ago

Ok so I fixed that inplace (sorry for mingling with your branch, I thought it would be the last thing), but there are several more places, in which I'm not so sure what the correct adaptions might be:

AndreBiedenkapp commented 5 years ago

Okay to keep us from having to mess too much with the code base we should cast every constant to One-valued Categoricals. That would automatically fix all issues above and for pyimp it would only mean that we need to pass a configspace object to fANOVA where we replaced constants with Categoricals. Do you agree @shukon? Will you handle that in https://github.com/automl/CAVE/issues/200?

shukon commented 5 years ago

@AndreBiedenkapp I'll give it a try.

shukon commented 5 years ago

I think it makes more sense to fix it in pimp, since other modules using pimp will run into the same issue sooner or later. So I opened a PR https://github.com/automl/ParameterImportance/pull/92... @AndreBiedenkapp can you check?

shukon commented 5 years ago

I think this is fixed.