automl / ParameterImportance

Parameter Importance Analysis Tool
http://www.ml4aad.org/
BSD 3-Clause "New" or "Revised" License
76 stars 19 forks source link

fANOVA visualization fails with bools in CategoricalHyperparameter #90

Closed shukon closed 4 years ago

shukon commented 5 years ago

If a categorical hyperparameter has as choice [True, False], everything works fine except for the visualization of pairwise marginals. I think this has something to do with conversion from runhistory data to fANOVA-data. I created a new branch with test-suites (just run nosetests) to recreate this.

Traceback (most recent call last):
  File "/home/shuki/Documents/ParameterImportance/test/test_configspace.py", line 41, in test_mixed_categorical
    self.imp.evaluate_scenario(['fanova'], 'test/configspace_cornercase')
  File "/home/shuki/Documents/ParameterImportance/pimp/importance/importance.py", line 503, in evaluate_scenario
    self.evaluator.plot_result(os.path.join(save_folder, self.evaluator.name.lower()), show=False)
  File "/home/shuki/Documents/ParameterImportance/pimp/evaluator/fanova.py", line 134, in plot_result
    vis.create_most_important_pairwise_marginal_plots(most_important_ones)
  File "/home/shuki/Documents/fanova/fanova/visualizer.py", line 386, in create_most_important_pairwise_marginal_plots
    self.plot_pairwise_marginal((param1, param2), show=False, three_d=three_d)
  File "/home/shuki/Documents/fanova/fanova/visualizer.py", line 180, in plot_pairwise_marginal
    plt.plot(zz[i], label='%s' %cat)
IndexError: index 2 is out of bounds for axis 0 with size 2