HunterMcGushion / hyperparameter_hunter

Easy hyperparameter optimization and automatic result saving across machine learning algorithms and libraries
MIT License
706 stars 100 forks source link

Tests for bug when using `BayesianOptPro` in exclusively-`Categorical` with `FeatureEngineer` #152

Closed HunterMcGushion closed 5 years ago

HunterMcGushion commented 5 years ago

Conditions to summon bug (assume a blood offering has already been made):

  1. Use BayesianOptPro
  2. Use exclusively Categorical dimensions
  3. At least one of the Categorical dimensions must be in FeatureEngineer

The above conditions will cause BayesianOptPro to break outright. However, problems still arise when omitting FeatureEngineer, while searching an exclusively multi-Categorical space. The effect of this bug is that saved experiment results are not properly marked as similar_experiments. Tests demonstrating the behavior of this second bug are added in 4b60f1a.

When the two xfail tests are marked as xpass, the bug is probably (hopefully) fixed

codecov[bot] commented 5 years ago

Codecov Report

Merging #152 into master will increase coverage by 0.13%. The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #152      +/-   ##
==========================================
+ Coverage   93.53%   93.67%   +0.13%     
==========================================
  Files          44       44              
  Lines        4393     4393              
==========================================
+ Hits         4109     4115       +6     
+ Misses        284      278       -6
Impacted Files Coverage Δ
hyperparameter_hunter/optimization_core.py 92.9% <0%> (-0.34%) :arrow_down:
hyperparameter_hunter/environment.py 98.72% <0%> (+0.42%) :arrow_up:
hyperparameter_hunter/utils/optimization_utils.py 90.28% <0%> (+3.42%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b4ff0cd...4b60f1a. Read the comment docs.

coveralls commented 5 years ago

Pull Request Test Coverage Report for Build 668


Files with Coverage Reduction New Missed Lines %
hyperparameter_hunter/optimization_core.py 2 93.92%
<!-- Total: 2 -->
Totals Coverage Status
Change from base Build 662: 0.2%
Covered Lines: 4119
Relevant Lines: 4393

💛 - Coveralls