Closed adodge closed 5 years ago
Relates to issue #92
Thanks for the PR! I tried running this on my machine, and the test seems to pass with and without the change. Is there something else I'm missing that will allow me to reproduce the error? I've been using sklearn version 0.19, upgraded to 0.20 to see if that made a difference, but it still passes. I've also tried a few different algorithms as the classifier, but no luck.
Also, if the error is related to partial_fit, I think it would make more sense to test it with an algorithm that supports partial_fit, such as sgd
instead of random_forest
. That algo also seems to run faster, which is nice when running a full suite of tests.
Hmm interesting! Switching to sgd
is a good point. Not sure how that ended up being a random forest. The test wasn't meant to be specific to any particular classifier. The error is happening when we try to call .max()
on a numpy array with None
s in, so the version of numpy might be important. Here's my pip list
:
atomicwrites (1.2.1)
attrs (18.2.0)
decorator (4.3.0)
future (0.16.0)
hpsklearn (0.0.3, /home/adodge/dev/oss/hyperopt/hyperopt-sklearn)
hyperopt (0.1.1)
joblib (0.12.5)
lockfile (0.12.2)
more-itertools (4.3.0)
networkx (2.2)
nose (1.3.7)
numpy (1.15.1)
pandas (0.23.4)
pip (9.0.1)
pluggy (0.7.1)
py (1.6.0)
pymongo (3.7.1)
pytest (3.8.0)
python-dateutil (2.7.3)
pytz (2018.5)
scikit-learn (0.19.2)
scipy (1.1.0)
setuptools (28.8.0)
six (1.11.0)
skdata (0.0.4, /home/adodge/dev/oss/hyperopt/skdata/build/py3k)
wheel (0.29.0)
I changed the classifier to SGD and ran test_estimator.py
with and without the fix:
With (https://github.com/adodge/hyperopt-sklearn/commit/ed38c602de55facf045d7eb64f1f83aaa0d533d4): test_results2.txt
Without (https://github.com/adodge/hyperopt-sklearn/commit/1612c0d85defd15fe0d6462d6b651ed009dc5bdf): test_results.txt
(Which unfortunately doesn't actually show the traceback to the line in question, but I found it by bodging in calls to traceback inside the subprocess.)
I'll try this with different versions of numpy when I get a chance.
Different versions of numpy don't seem to matter. I tried all the major versions back to 1.11 and they all fail the test for me without the fix. How odd!
Just tried making a fresh virtualenv, and now its working correctly (i.e. old code fails, new code doesn't). Must be something weird with my environment.
Great work! Going to merge this now
Using the CV options without
use_partial_fit
causes an error becausecv_n_iters
gets filled withNone
, which can't be passed tomax()
. This is a patch that fixes this, along with a test that used to fail, but now doesn't.