search

ScottfreeLLC / AlphaPy

Python AutoML for Trading Systems and Sports Betting
Apache License 2.0
1.14k stars 207 forks source link

ValueError: model.yml features:encoding:type target unrecognized #31

Closed jim-schwoebel closed 4 years ago

jim-schwoebel commented 4 years ago

I'm getting this error when running the examples.

Any ideas?

Traceback (most recent call last):
  File "/usr/local/bin/alphapy", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 471, in main
    specs = get_model_config()
  File "/usr/local/lib/python3.7/site-packages/alphapy/model.py", line 270, in get_model_config
    raise ValueError("model.yml features:encoding:type %s unrecognized" % encoder)
ValueError: model.yml features:encoding:type target unrecognized
jim-schwoebel commented 4 years ago

This was for the NCAAB example but seems to error in the same way across all examples.

jim-schwoebel commented 4 years ago

Operating system info below.

platform: 'Darwin-18.7.0-x86_64-i386-64bit' release: '18.7.0' version: 'Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64'

mrconway commented 4 years ago

Sure, could you check which version of AlphaPy you're running? Run the following command:

(base) localhost:AlphaPy markconway$ pip show alphapy Name: alphapy Version: 2.4.2

You may have to update dependencies as well:

$ pip install -U alphapy

Then try running the examples again. Thank you!

jim-schwoebel commented 4 years ago

Will do! I think the problem here was that I had a virtual environment running but didn't actually install alphapy - it was working because I was in the folder when I cloned the repo and had all the dependencies installed.

Nonetheless, I did install alphapy (version 2.4.2) and then got this error when running an example:

 File "/usr/local/bin/alphapy", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 503, in main
    model = main_pipeline(model)
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 426, in main_pipeline
    model = training_pipeline(model)
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 252, in training_pipeline
    model = first_fit(model, algo, est)
  File "/usr/local/lib/python3.7/site-packages/alphapy/model.py", line 704, in first_fit
    if hasattr(est, "coef_"):
  File "/usr/local/lib/python3.7/site-packages/xgboost/sklearn.py", line 716, in coef_
    coef = np.array(json.loads(b.get_dump(dump_format='json')[0])['weight'])
KeyError: 'weight'
jim-schwoebel commented 4 years ago

Full error log below:

(env) jimschwoebel@Jims-MBP kaggle % alphapy                      
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/externals/six.py:31: FutureWarning: The module is deprecated in version 0.21 and will be removed in version 0.23 since we've dropped support for Python 2.7. Please rely on the official version of six (https://pypi.org/project/six/).
  "(https://pypi.org/project/six/).", FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.neighbors.base module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.neighbors. Anything that cannot be imported from sklearn.neighbors is now part of the private API.
  warnings.warn(message, FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.bagging module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.base module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.forest module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=FutureWarning)
/usr/local/lib/python3.7/site-packages/pandas_datareader/compat/__init__.py:7: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  from pandas.util.testing import assert_frame_equal
Using TensorFlow backend.
None
/usr/local/lib/python3.7/site-packages/alphapy/__main__.py:181: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_train[target] = y_train
[0] validation_0-auc:0.87057    validation_1-auc:0.80633
Multiple eval metrics have been passed: 'validation_1-auc' will be used for early stopping.

Will train until validation_1-auc hasn't improved in 20 rounds.
[1] validation_0-auc:0.87073    validation_1-auc:0.80618
[2] validation_0-auc:0.87068    validation_1-auc:0.80618
[3] validation_0-auc:0.87437    validation_1-auc:0.80774
[4] validation_0-auc:0.87326    validation_1-auc:0.80695
[5] validation_0-auc:0.87321    validation_1-auc:0.80695
[6] validation_0-auc:0.87457    validation_1-auc:0.80725
[7] validation_0-auc:0.87454    validation_1-auc:0.80725
[8] validation_0-auc:0.87451    validation_1-auc:0.80725
[9] validation_0-auc:0.87448    validation_1-auc:0.80725
[10]    validation_0-auc:0.87445    validation_1-auc:0.80725
[11]    validation_0-auc:0.87411    validation_1-auc:0.81250
[12]    validation_0-auc:0.87415    validation_1-auc:0.81312
[13]    validation_0-auc:0.87415    validation_1-auc:0.81316
[14]    validation_0-auc:0.87415    validation_1-auc:0.81312
[15]    validation_0-auc:0.87652    validation_1-auc:0.82877
[16]    validation_0-auc:0.88211    validation_1-auc:0.83053
[17]    validation_0-auc:0.88293    validation_1-auc:0.83158
[18]    validation_0-auc:0.88423    validation_1-auc:0.86282
[19]    validation_0-auc:0.88428    validation_1-auc:0.86242
[20]    validation_0-auc:0.89817    validation_1-auc:0.85788
[21]    validation_0-auc:0.91166    validation_1-auc:0.85817
[22]    validation_0-auc:0.91152    validation_1-auc:0.85047
[23]    validation_0-auc:0.91203    validation_1-auc:0.85615
[24]    validation_0-auc:0.91439    validation_1-auc:0.85750
[25]    validation_0-auc:0.91421    validation_1-auc:0.85034
[26]    validation_0-auc:0.91684    validation_1-auc:0.83964
[27]    validation_0-auc:0.91439    validation_1-auc:0.85674
[28]    validation_0-auc:0.91550    validation_1-auc:0.85151
[29]    validation_0-auc:0.91861    validation_1-auc:0.84755
[30]    validation_0-auc:0.92000    validation_1-auc:0.84272
[31]    validation_0-auc:0.92063    validation_1-auc:0.84161
[32]    validation_0-auc:0.92029    validation_1-auc:0.83888
[33]    validation_0-auc:0.92132    validation_1-auc:0.83777
[34]    validation_0-auc:0.92151    validation_1-auc:0.84202
[35]    validation_0-auc:0.92224    validation_1-auc:0.84067
[36]    validation_0-auc:0.92429    validation_1-auc:0.83781
[37]    validation_0-auc:0.92457    validation_1-auc:0.84148
[38]    validation_0-auc:0.92415    validation_1-auc:0.84333
Stopping. Best iteration:
[18]    validation_0-auc:0.88423    validation_1-auc:0.86282

/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py:14: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing as tm
/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py:14: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing as tm
/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py:14: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing as tm
Traceback (most recent call last):
  File "/usr/local/bin/alphapy", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 503, in main
    model = main_pipeline(model)
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 426, in main_pipeline
    model = training_pipeline(model)
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 252, in training_pipeline
    model = first_fit(model, algo, est)
  File "/usr/local/lib/python3.7/site-packages/alphapy/model.py", line 704, in first_fit
    if hasattr(est, "coef_"):
  File "/usr/local/lib/python3.7/site-packages/xgboost/sklearn.py", line 716, in coef_
    coef = np.array(json.loads(b.get_dump(dump_format='json')[0])['weight'])
KeyError: 'weight'
jim-schwoebel commented 4 years ago

Looking a little deeper - this looks like it may be something with XGboost: 

>>> xgboost.__version__
'1.0.2'

I'm using version 1.0.2 but will keep digging

mrconway commented 4 years ago

I remember this issue. Try running with xgboost 0.90. We're going to fix this with dynamic import of these external libraries.

jim-schwoebel commented 4 years ago

Aight - sounds good will do now.

jim-schwoebel commented 4 years ago

Ok cool. I installed XGBoost 0.90 + re-ran. Still seems like the same error is happening.

I'm hoping to extend alphapy a bit and work it into a general-purpose ML framework I'm building, so will keep trying to get this all working on my cpu - or try a linux OS.

A log is below if it helps.

(env) jimschwoebel@Jims-MBP kaggle % pip3 install xgboost==0.90 
Collecting xgboost==0.90
  Using cached xgboost-0.90.tar.gz (676 kB)
Requirement already satisfied: numpy in /Users/jimschwoebel/Desktop/allie/training/helpers/AlphaPy/alphapy/env/lib/python3.7/site-packages (from xgboost==0.90) (1.18.2)
Requirement already satisfied: scipy in /Users/jimschwoebel/Desktop/allie/training/helpers/AlphaPy/alphapy/env/lib/python3.7/site-packages (from xgboost==0.90) (1.4.1)
Building wheels for collected packages: xgboost
  Building wheel for xgboost (setup.py) ... done
  Created wheel for xgboost: filename=xgboost-0.90-cp37-cp37m-macosx_10_15_x86_64.whl size=1710236 sha256=7e40c5e7e252782336de7f97235b1834be8988a1109f930ffddc74d535005c4a
  Stored in directory: /Users/jimschwoebel/Library/Caches/pip/wheels/66/5a/1f/f1d545c72a3d2303225d4eee7bfb5df1edb5054b14658980fd
Successfully built xgboost
Installing collected packages: xgboost
  Attempting uninstall: xgboost
    Found existing installation: xgboost 1.0.0
    Uninstalling xgboost-1.0.0:
      Successfully uninstalled xgboost-1.0.0
Successfully installed xgboost-0.90

Then I ran

(env) jimschwoebel@Jims-MBP kaggle % alphapy                   
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/externals/six.py:31: FutureWarning: The module is deprecated in version 0.21 and will be removed in version 0.23 since we've dropped support for Python 2.7. Please rely on the official version of six (https://pypi.org/project/six/).
  "(https://pypi.org/project/six/).", FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.neighbors.base module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.neighbors. Anything that cannot be imported from sklearn.neighbors is now part of the private API.
  warnings.warn(message, FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.bagging module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.base module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.forest module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
/Users/jimschwoebel/Library/Python/3.7/lib/python/site-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=FutureWarning)
/usr/local/lib/python3.7/site-packages/pandas_datareader/compat/__init__.py:7: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  from pandas.util.testing import assert_frame_equal
Using TensorFlow backend.
None
/usr/local/lib/python3.7/site-packages/alphapy/__main__.py:181: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_train[target] = y_train
[0] validation_0-auc:0.87057    validation_1-auc:0.80633
Multiple eval metrics have been passed: 'validation_1-auc' will be used for early stopping.

Will train until validation_1-auc hasn't improved in 20 rounds.
[1] validation_0-auc:0.87073    validation_1-auc:0.80618
[2] validation_0-auc:0.87068    validation_1-auc:0.80618
[3] validation_0-auc:0.87437    validation_1-auc:0.80774
[4] validation_0-auc:0.87326    validation_1-auc:0.80695
[5] validation_0-auc:0.87321    validation_1-auc:0.80695
[6] validation_0-auc:0.87457    validation_1-auc:0.80725
[7] validation_0-auc:0.87454    validation_1-auc:0.80725
[8] validation_0-auc:0.87451    validation_1-auc:0.80725
[9] validation_0-auc:0.87448    validation_1-auc:0.80725
[10]    validation_0-auc:0.87445    validation_1-auc:0.80725
[11]    validation_0-auc:0.87411    validation_1-auc:0.81250
[12]    validation_0-auc:0.87415    validation_1-auc:0.81312
[13]    validation_0-auc:0.87415    validation_1-auc:0.81316
[14]    validation_0-auc:0.87415    validation_1-auc:0.81312
[15]    validation_0-auc:0.87652    validation_1-auc:0.82877
[16]    validation_0-auc:0.88211    validation_1-auc:0.83053
[17]    validation_0-auc:0.88293    validation_1-auc:0.83158
[18]    validation_0-auc:0.88423    validation_1-auc:0.86282
[19]    validation_0-auc:0.88428    validation_1-auc:0.86242
[20]    validation_0-auc:0.89817    validation_1-auc:0.85788
[21]    validation_0-auc:0.91166    validation_1-auc:0.85817
[22]    validation_0-auc:0.91152    validation_1-auc:0.85047
[23]    validation_0-auc:0.91203    validation_1-auc:0.85615
[24]    validation_0-auc:0.91439    validation_1-auc:0.85750
[25]    validation_0-auc:0.91421    validation_1-auc:0.85034
[26]    validation_0-auc:0.91684    validation_1-auc:0.83964
[27]    validation_0-auc:0.91439    validation_1-auc:0.85674
[28]    validation_0-auc:0.91550    validation_1-auc:0.85151
[29]    validation_0-auc:0.91861    validation_1-auc:0.84755
[30]    validation_0-auc:0.92000    validation_1-auc:0.84272
[31]    validation_0-auc:0.92063    validation_1-auc:0.84161
[32]    validation_0-auc:0.92029    validation_1-auc:0.83888
[33]    validation_0-auc:0.92132    validation_1-auc:0.83777
[34]    validation_0-auc:0.92151    validation_1-auc:0.84202
[35]    validation_0-auc:0.92224    validation_1-auc:0.84067
[36]    validation_0-auc:0.92429    validation_1-auc:0.83781
[37]    validation_0-auc:0.92457    validation_1-auc:0.84148
[38]    validation_0-auc:0.92415    validation_1-auc:0.84333
Stopping. Best iteration:
[18]    validation_0-auc:0.88423    validation_1-auc:0.86282

/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py:14: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing as tm
/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py:14: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing as tm
/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py:14: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing as tm
Traceback (most recent call last):
  File "/usr/local/bin/alphapy", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 503, in main
    model = main_pipeline(model)
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 426, in main_pipeline
    model = training_pipeline(model)
  File "/usr/local/lib/python3.7/site-packages/alphapy/__main__.py", line 252, in training_pipeline
    model = first_fit(model, algo, est)
  File "/usr/local/lib/python3.7/site-packages/alphapy/model.py", line 704, in first_fit
    if hasattr(est, "coef_"):
  File "/usr/local/lib/python3.7/site-packages/xgboost/sklearn.py", line 716, in coef_
    coef = np.array(json.loads(b.get_dump(dump_format='json')[0])['weight'])
KeyError: 'weight'
mrconway commented 4 years ago

Okay, you got by the target encoding error, looks like a real bug when trying to access the XGBoost coefficients from sklearn.py. Thank you.

jim-schwoebel commented 4 years ago

np - will let you know if I figure it out

jim-schwoebel commented 4 years ago

If this helps, these are the current requirements.txt I have after running pip freeze >> requirements.txt

absl-py==0.9.0
alphapy==2.4.2
appnope==0.1.0
arrow==0.13.0
astor==0.8.1
backcall==0.1.0
bokeh==2.0.0
cachetools==4.0.0
category-encoders==2.1.0
certifi==2019.11.28
chardet==3.0.4
cycler==0.10.0
decorator==4.4.2
empyrical==0.5.3
gast==0.2.2
google-auth==1.12.0
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio==1.27.2
h5py==2.10.0
idna==2.9
iexfinance==0.4.3
imbalanced-learn==0.6.2
ipython==7.13.0
ipython-genutils==0.2.0
jedi==0.16.0
Jinja2==2.11.1
joblib==0.14.1
Keras==2.3.1
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
lxml==4.5.0
Markdown==3.2.1
MarkupSafe==1.1.1
matplotlib==3.2.1
numpy==1.18.2
oauthlib==3.1.0
opt-einsum==3.2.0
packaging==20.3
pandas==1.0.3
pandas-datareader==0.8.0
parso==0.6.2
patsy==0.5.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.0.0
prompt-toolkit==3.0.5
protobuf==3.11.3
ptyprocess==0.6.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyfolio==0.9.0
Pygments==2.6.1
pyparsing==2.4.6
python-dateutil==2.8.1
pytz==2019.3
PyYAML==5.3.1
requests==2.23.0
requests-oauthlib==1.3.0
rsa==4.0
scikit-learn==0.22.2.post1
scipy==1.4.1
seaborn==0.10.0
six==1.14.0
statsmodels==0.11.1
tensorboard==2.1.1
tensorflow==2.1.0
tensorflow-estimator==2.1.0
termcolor==1.1.0
tornado==6.0.4
traitlets==4.3.3
typing-extensions==3.7.4.1
urllib3==1.25.8
wcwidth==0.1.9
Werkzeug==1.0.0
wrapt==1.12.1
xgboost==0.90
mrconway commented 4 years ago

Okay, we're using the following:

(base) localhost:AlphaPy markconway$ pip show scikit-learn Name: scikit-learn Version: 0.22.1

jim-schwoebel commented 4 years ago

ok changing to that

jim-schwoebel commented 4 years ago

Still getting the same error

irst_fit
    if hasattr(est, "coef_"):
  File "/usr/local/lib/python3.7/site-packages/xgboost/sklearn.py", line 716, in coef_
    coef = np.array(json.loads(b.get_dump(dump_format='json')[0])['weight'])
KeyError: 'weight'
jim-schwoebel commented 4 years ago

Okay cool - going to go grab some dinner. I'll keep trying to dig around here and post any updates if I figure it out

jim-schwoebel commented 4 years ago

Wasn't able to figure this out, so going to try to work with a Linux OS over next few days to get stuff running on this repo.

Thanks for making a rich library here; looks like a great tool among others in the AutoML field.

mrconway commented 4 years ago

Thanks for your kind words Jim, and let us know of any enhancements you'd like to see.

jim-schwoebel commented 4 years ago

Hey - trying again here.

I loaded up Linux / Ubuntu on WIndows and I'm on Python 3.

I got the first example to work again and have a project with the config files. 

jim@DESKTOP-MBFTMVI:/mnt/c/users/jimsc/desktop/allie/models/Battlecry_Cashregister_librosa_features_alphapy$ tree
.
├── alphapy.log
├── config
│   ├── algos.yml
│   └── model.yml
├── data
├── input
│   ├── test.csv
│   ├── test_20200422.csv
│   ├── train.csv
│   └── train_20200422.csv
├── model
├── output
└── plots

This was with some custom data in a .CSV file (train and test). I have all these files attached Battlecry_Cashregister_librosa_features_alphapy.zip

When I cd into the folder and run alphapy I get stuck at this step:

jim@DESKTOP-MBFTMVI:/mnt/c/users/jimsc/desktop/allie/models/Battlecry_Cashregister_librosa_features_alphapy$ alphapy
/home/jim/.local/lib/python3.6/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.neighbors.base module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.neighbors. Anything that cannot be imported from sklearn.neighbors is now part of the private API.
  warnings.warn(message, FutureWarning)
/home/jim/.local/lib/python3.6/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.bagging module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
/home/jim/.local/lib/python3.6/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.base module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
/home/jim/.local/lib/python3.6/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.ensemble.forest module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.ensemble. Anything that cannot be imported from sklearn.ensemble is now part of the private API.
  warnings.warn(message, FutureWarning)
Using TensorFlow backend.
/home/jim/.local/lib/python3.6/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.utils.testing module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.utils. Anything that cannot be imported from sklearn.utils is now part of the private API.
  warnings.warn(message, FutureWarning)
/home/jim/.local/lib/python3.6/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.metrics.classification module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.metrics. Anything that cannot be imported from sklearn.metrics is now part of the private API.
  warnings.warn(message, FutureWarning)
/home/jim/.local/lib/python3.6/site-packages/pandas_datareader/compat/__init__.py:7: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  from pandas.util.testing import assert_frame_equal
None
[04/22/20 18:35:25] INFO        ********************************************************************************
[04/22/20 18:35:25] INFO        AlphaPy Start
[04/22/20 18:35:25] INFO        ********************************************************************************
[04/22/20 18:35:25] INFO        Model Configuration
[04/22/20 18:35:25] INFO        No transforms Found
[04/22/20 18:35:25] INFO        MODEL PARAMETERS:
[04/22/20 18:35:25] INFO        algorithms        = ['RF', 'XGB']
[04/22/20 18:35:25] INFO        calibration       = False
[04/22/20 18:35:25] INFO        cal_type          = isotonic
[04/22/20 18:35:25] INFO        calibration_plot  = False
[04/22/20 18:35:25] INFO        clustering        = False
[04/22/20 18:35:25] INFO        cluster_inc       = 3
[04/22/20 18:35:25] INFO        cluster_max       = 30
[04/22/20 18:35:25] INFO        cluster_min       = 3
[04/22/20 18:35:25] INFO        confusion_matrix  = True
[04/22/20 18:35:25] INFO        counts            = False
[04/22/20 18:35:25] INFO        cv_folds          = 3
[04/22/20 18:35:25] INFO        directory         = .
[04/22/20 18:35:25] INFO        extension         = csv
[04/22/20 18:35:25] INFO        drop              = ['Unnamed: 0']
[04/22/20 18:35:25] INFO        encoder           = <Encoders.target: 14>
[04/22/20 18:35:25] INFO        esr               = 30
[04/22/20 18:35:25] INFO        factors           = None
[04/22/20 18:35:25] INFO        features [X]      = *
[04/22/20 18:35:25] INFO        feature_selection = False
[04/22/20 18:35:25] INFO        fs_percentage     = 50
[04/22/20 18:35:25] INFO        fs_score_func     = <function f_classif at 0x7fa520dd5048>
[04/22/20 18:35:25] INFO        fs_uni_grid       = [5, 10, 15, 20, 25]
[04/22/20 18:35:25] INFO        grid_search       = True
[04/22/20 18:35:25] INFO        gs_iters          = 50
[04/22/20 18:35:25] INFO        gs_random         = True
[04/22/20 18:35:25] INFO        gs_sample         = False
[04/22/20 18:35:25] INFO        gs_sample_pct     = 0.250000
[04/22/20 18:35:25] INFO        importances       = True
[04/22/20 18:35:25] INFO        interactions      = True
[04/22/20 18:35:25] INFO        isomap            = False
[04/22/20 18:35:25] INFO        iso_components    = 2
[04/22/20 18:35:25] INFO        iso_neighbors     = 5
[04/22/20 18:35:25] INFO        isample_pct       = 5
[04/22/20 18:35:25] INFO        learning_curve    = True
[04/22/20 18:35:25] INFO        logtransform      = False
[04/22/20 18:35:25] INFO        lv_remove         = True
[04/22/20 18:35:25] INFO        lv_threshold      = 0.100000
[04/22/20 18:35:25] INFO        model_type        = <ModelType.classification: 1>
[04/22/20 18:35:25] INFO        n_estimators      = 201
[04/22/20 18:35:25] INFO        n_jobs            = -1
[04/22/20 18:35:25] INFO        ngrams_max        = 1
[04/22/20 18:35:25] INFO        numpy             = False
[04/22/20 18:35:25] INFO        pca               = False
[04/22/20 18:35:25] INFO        pca_inc           = 3
[04/22/20 18:35:25] INFO        pca_max           = 15
[04/22/20 18:35:25] INFO        pca_min           = 3
[04/22/20 18:35:25] INFO        pca_whiten        = False
[04/22/20 18:35:25] INFO        poly_degree       = 2
[04/22/20 18:35:25] INFO        pvalue_level      = 0.010000
[04/22/20 18:35:25] INFO        rfe               = True
[04/22/20 18:35:25] INFO        rfe_step          = 5
[04/22/20 18:35:25] INFO        roc_curve         = True
[04/22/20 18:35:25] INFO        rounding          = 3
[04/22/20 18:35:25] INFO        sampling          = False
[04/22/20 18:35:25] INFO        sampling_method   = <SamplingMethod.under_random: 12>
[04/22/20 18:35:25] INFO        sampling_ratio    = 0.000000
[04/22/20 18:35:25] INFO        scaler_option     = True
[04/22/20 18:35:25] INFO        scaler_type       = <Scalers.standard: 2>
[04/22/20 18:35:25] INFO        scipy             = False
[04/22/20 18:35:25] INFO        scorer            = roc_auc
[04/22/20 18:35:25] INFO        seed              = 13201
[04/22/20 18:35:25] INFO        sentinel          = -1
[04/22/20 18:35:25] INFO        separator         = ,
[04/22/20 18:35:25] INFO        shuffle           = True
[04/22/20 18:35:25] INFO        split             = 0.400000
[04/22/20 18:35:25] INFO        submission_file   = Battlecry_Cashregister_librosa_features_alphapy
[04/22/20 18:35:25] INFO        submit_probas     = False
[04/22/20 18:35:25] INFO        target [y]        = class
[04/22/20 18:35:25] INFO        target_value      = 1
[04/22/20 18:35:25] INFO        transforms        = None
[04/22/20 18:35:25] INFO        tsne              = False
[04/22/20 18:35:25] INFO        tsne_components   = 2
[04/22/20 18:35:25] INFO        tsne_learn_rate   = 1000.000000
[04/22/20 18:35:25] INFO        tsne_perplexity   = 30.000000
[04/22/20 18:35:25] INFO        vectorize         = False
[04/22/20 18:35:25] INFO        verbosity         = 0
[04/22/20 18:35:25] INFO        Creating Model
[04/22/20 18:35:25] INFO        Calling Pipeline
[04/22/20 18:35:25] INFO        Training Pipeline
[04/22/20 18:35:25] INFO        Loading Data
[04/22/20 18:35:25] INFO        Loading data from ./input/train.csv
[04/22/20 18:35:25] INFO        Found target class in data frame
[04/22/20 18:35:25] INFO        Labels (y) found for Partition.train
[04/22/20 18:35:25] INFO        Loading Data
[04/22/20 18:35:25] INFO        Loading data from ./input/test.csv
[04/22/20 18:35:25] INFO        Found target class in data frame
[04/22/20 18:35:25] INFO        Labels (y) found for Partition.test
[04/22/20 18:35:25] INFO        Test Labels Found
[04/22/20 18:35:25] INFO        Saving New Features in Model
[04/22/20 18:35:25] INFO        Original Feature Statistics
[04/22/20 18:35:25] INFO        Number of Training Rows    : 85
[04/22/20 18:35:25] INFO        Number of Training Columns : 187
[04/22/20 18:35:25] INFO        Unique Training Values for class : [0 1]
[04/22/20 18:35:25] INFO        Unique Training Counts for class : [43 42]
[04/22/20 18:35:25] INFO        Number of Testing Rows     : 29
[04/22/20 18:35:25] INFO        Number of Testing Columns  : 187
[04/22/20 18:35:25] INFO        Unique Testing Values for class : [0 1]
[04/22/20 18:35:25] INFO        Unique Testing Counts for class : [14 15]
[04/22/20 18:35:25] INFO        Original Features : Index(['Unnamed: 0', 'onset_length', 'onset_detect_mean', 'onset_detect_std',
       'onset_detect_maxv', 'onset_detect_minv', 'onset_detect_median',
       'tempo', 'onset_strength_mean', 'onset_strength_std',
       ...
       'spectral_rolloff_median', 'zero_crossings_mean', 'zero_crossings_std',
       'zero_crossings_maxv', 'zero_crossings_minv', 'zero_crossings_median',
       'RMSE_mean', 'RMSE_std', 'RMSE_maxv', 'RMSE_minv'],
      dtype='object', length=187)
[04/22/20 18:35:25] INFO        Feature Count     : 187
[04/22/20 18:35:25] INFO        Applying transforms
[04/22/20 18:35:25] INFO        No transforms Specified
[04/22/20 18:35:25] INFO        New Feature Count : 187
[04/22/20 18:35:25] INFO        Dropping Features: ['Unnamed: 0']
[04/22/20 18:35:25] INFO        Original Feature Count : 187
[04/22/20 18:35:25] INFO        Reduced Feature Count  : 186
/home/jim/.local/lib/python3.6/site-packages/alphapy/__main__.py:181: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_train[target] = y_train
[04/22/20 18:35:25] INFO        Writing data frame to ./input/train_20200422.csv
/home/jim/.local/lib/python3.6/site-packages/alphapy/__main__.py:186: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_test[target] = y_test
[04/22/20 18:35:25] INFO        Writing data frame to ./input/test_20200422.csv
[04/22/20 18:35:25] INFO        Creating Cross-Tabulations
Traceback (most recent call last):
  File "/home/jim/.local/bin/alphapy", line 8, in <module>
    sys.exit(main())
  File "/home/jim/.local/lib/python3.6/site-packages/alphapy/__main__.py", line 503, in main
    model = main_pipeline(model)
  File "/home/jim/.local/lib/python3.6/site-packages/alphapy/__main__.py", line 426, in main_pipeline
    model = training_pipeline(model)
  File "/home/jim/.local/lib/python3.6/site-packages/alphapy/__main__.py", line 193, in training_pipeline
    create_crosstabs(model)
  File "/home/jim/.local/lib/python3.6/site-packages/alphapy/features.py", line 506, in create_crosstabs
    if fname in factors:

Any ideas on how to fix this? Is this a problem with the way I formatted the .CSV files?

jim-schwoebel commented 4 years ago

I'm running Python 3.6.9 on Ubuntu (shown below):

release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.4 LTS"
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
jim-schwoebel commented 4 years ago

Nevermind - I got it all working. Just updated a few dependencies here and the YAML file and it worked fine.