Closed franckjay closed 2 years ago
Trying to run inference on a new dataset with the following installed packages:
Package Version ---------------------------------- ------------------- absl-py 1.0.0 aiobotocore 1.3.0 aiohttp 3.8.1 aioitertools 0.7.1 aiosignal 1.2.0 alabaster 0.7.12 anyio 3.2.0 argh 0.26.2 argon2-cffi 20.1.0 asn1crypto 1.4.0 astor 0.8.1 astroid 2.5.6 astropy 4.1 async-generator 1.10 async-timeout 4.0.1 asynctest 0.13.0 atomicwrites 1.4.0 attrs 21.2.0 autopep8 1.5.7 autovizwidget 0.19.1 awscli 1.24.10 Babel 2.9.1 backcall 0.2.0 backports.functools-lru-cache 1.6.4 backports.shutil-get-terminal-size 1.0.0 base 1.0.4 bcrypt 4.0.0 beautifulsoup4 4.9.3 bitarray 2.1.0 bkcharts 0.2 bleach 4.1.0 blis 0.7.4 bokeh 2.3.3 boto 2.49.0 boto3 1.23.10 botocore 1.26.10 Bottleneck 1.3.2 brotlipy 0.7.0 bz2file 0.98 cached-property 1.5.2 cachetools 4.2.2 catalogue 2.0.4 certifi 2020.6.20 cffi 1.14.5 chardet 4.0.0 charset-normalizer 2.0.9 click 8.0.1 cloudpickle 1.6.0 clyent 1.2.2 colorama 0.4.3 contextlib2 21.6.0 contextvars 2.4 coverage 6.2 cryptography 3.4.7 cycler 0.11.0 cymem 2.0.5 Cython 0.29.6 cytoolz 0.11.0 dask 2021.3.0 dataclasses 0.8 decorator 5.0.9 defusedxml 0.7.1 diff-match-patch 20200713 dill 0.3.4 distributed 2021.3.0 distro 1.7.0 dm-sonnet 1.29 docker 5.0.3 docker-compose 1.29.2 dockerpty 0.4.1 docopt 0.6.2 docutils 0.15.2 entrypoints 0.3 environment-kernels 1.1.1 et-xmlfile 1.1.0 fastai 1.0.61 fastcache 1.1.0 fastprogress 1.0.0 filelock 3.0.12 flake8 3.9.2 Flask 2.0.1 Flask-Cors 3.0.10 frozenlist 1.2.0 fsspec 2021.4.0 future 0.18.2 gast 0.5.3 gevent 21.1.2 glob2 0.7 gmpy2 2.1.0b5 google-api-core 1.14.3 google-auth 1.30.2 google-cloud-core 1.1.0 google-cloud-storage 1.20.0 google-pasta 0.2.0 google-resumable-media 0.3.3 googleapis-common-protos 1.53.0 greenlet 1.1.0 grpcio 1.44.0 gssapi 1.7.2 h5py 2.9.0 hdijupyterutils 0.19.1 HeapDict 1.0.1 html5lib 1.1 idna 2.10 idna-ssl 1.1.0 imageio 2.9.0 imagesize 1.2.0 immutables 0.15 importlib-metadata 4.8.3 iniconfig 1.1.1 intervaltree 3.1.0 ipykernel 5.5.5 ipyparallel 6.3.0 ipython 7.16.1 ipython_genutils 0.2.0 ipywidgets 7.6.3 isort 5.9.1 itsdangerous 2.0.1 jdcal 1.4.1 jedi 0.17.2 jeepney 0.6.0 Jinja2 3.0.1 jmespath 0.10.0 joblib 1.1.0 json-rpc 1.12.2 json5 0.9.5 jsonschema 3.2.0 jupyter 1.0.0 jupyter-client 6.1.12 jupyter-console 6.4.0 jupyter-core 4.7.1 jupyter-server 1.8.0 jupyterlab 3.2.4 jupyterlab-pygments 0.1.2 jupyterlab-server 2.6.0 jupyterlab-widgets 1.0.0 Keras 2.2.4 Keras-Applications 1.0.7 Keras-Preprocessing 1.0.9 keyring 23.0.1 kiwisolver 1.3.1 krb5 0.2.0 lazy-object-proxy 1.6.0 libarchive-c 3.1 llvmlite 0.36.0 locket 0.2.1 lxml 4.6.3 Markdown 3.3.6 MarkupSafe 2.0.1 matplotlib 3.0.3 mccabe 0.6.1 mistune 0.8.4 mkl-fft 1.0.14 mkl-random 1.2.2 mkl-service 2.4.0 mock 4.0.3 more-itertools 8.8.0 mpmath 1.2.1 msgpack 1.0.2 multidict 5.1.0 multipledispatch 0.6.0 multiprocess 0.70.12.2 murmurhash 1.0.5 nb-conda 2.2.1 nb-conda-kernels 2.3.1 nbclassic 0.3.1 nbclient 0.5.3 nbconvert 6.0.7 nbformat 5.1.3 nest-asyncio 1.5.1 networkx 2.5 nltk 3.4.5 nose 1.3.7 notebook 6.4.6 numba 0.53.1 numexpr 2.7.3 numpy 1.16.2 numpydoc 1.1.0 olefile 0.46 onnx 1.10.0 opencv-python 4.5.1.48 openpyxl 3.0.7 packaging 21.3 pandas 0.24.2 pandocfilters 1.4.3 paramiko 2.11.0 parso 0.7.1 partd 1.2.0 path 16.0.0 pathlib2 2.3.5 pathos 0.2.8 pathtools 0.1.2 pathy 0.5.2 patsy 0.5.1 pep8 1.7.1 pexpect 4.8.0 pickleshare 0.7.5 Pillow 8.4.0 pip 21.3.1 pkginfo 1.7.0 plotly 5.4.0 pluggy 0.13.1 ply 3.11 pox 0.3.0 ppft 1.6.6.4 preshed 3.0.5 prometheus-client 0.11.0 prompt-toolkit 3.0.19 protobuf 3.19.4 protobuf3-to-dict 0.1.5 psutil 5.8.0 psycopg2 2.7.5 ptyprocess 0.7.0 py 1.10.0 py4j 0.10.7 pyaml 21.10.1 pyarrow 6.0.1 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycodestyle 2.7.0 pycosat 0.6.3 pycparser 2.20 pycrypto 2.6.1 pycurl 7.43.0.6 pydantic 1.8.2 pydocstyle 6.1.1 pyflakes 2.3.1 pyfunctional 1.4.3 pygal 2.4.0 Pygments 2.9.0 pyinstrument 3.4.2 pyinstrument-cext 0.2.4 pykerberos 1.2.1 pylint 2.8.3 PyNaCl 1.5.0 pynvml 11.0.0 pyodbc 4.0.30 pyOpenSSL 20.0.1 pyparsing 3.0.7 PyQt5 5.12.3 PyQt5_sip 4.19.18 PyQtChart 5.12 PyQtWebEngine 5.12.1 pyrsistent 0.17.3 PySocks 1.7.1 pyspark 2.4.0 pyspnego 0.3.1 pytest 6.2.4 python-dateutil 2.8.2 python-dotenv 0.20.0 python-jsonrpc-server 0.4.0 python-language-server 0.11.1 pytz 2021.3 pyu2f 0.1.5 PyWavelets 1.1.1 pyxdg 0.27 PyYAML 6.0 pyzmq 22.1.0 QDarkStyle 3.0.2 QtAwesome 1.0.2 qtconsole 5.1.0 QtPy 1.9.0 regex 2021.4.4 requests 2.25.1 requests-kerberos 0.14.0 rope 0.19.0 rsa 4.7.2 Rtree 0.9.7 ruamel-yaml-conda 0.15.100 s3fs 2021.4.0 s3transfer 0.5.0 sagemaker 2.106.0 sagemaker-pyspark 1.4.2 scikit-image 0.17.2 scikit-learn 0.20.3 scikit-optimize 0.9.0 scikit-surprise 1.1.1 scipy 1.2.1 seaborn 0.9.0 SecretStorage 3.3.1 semantic-version 2.9.0 Send2Trash 1.8.0 setuptools 52.0.0.post20210125 shap 0.40.0 shellingham 1.4.0 simplegeneric 0.8.1 singledispatch 0.0.0 sip 4.19.25 six 1.16.0 sklearn 0.0 slicer 0.0.7 smart-open 2.2.1 smclarify 0.1 smdebug 1.0.12 smdebug-rulesconfig 1.0.1 sniffio 1.2.0 snowballstemmer 2.1.0 sortedcollections 2.1.0 sortedcontainers 2.4.0 soupsieve 2.2.1 spacy 3.0.6 spacy-legacy 3.0.6 sparkmagic 0.15.0 Sphinx 4.0.2 sphinxcontrib-applehelp 1.0.2 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 sphinxcontrib-websupport 1.2.4 spyder 3.3.6 spyder-kernels 0.5.2 SQLAlchemy 1.4.18 srsly 2.4.1 statsmodels 0.12.2 sympy 1.8 tables 3.6.1 tabulate 0.8.9 TBB 0.1 tblib 1.7.0 tenacity 8.0.1 tensorboard 1.13.1 tensorflow 1.13.1 tensorflow-estimator 1.13.0 tensorflow-probability 0.6.0 termcolor 1.1.0 terminado 0.10.1 testpath 0.5.0 texttable 1.6.4 thinc 8.0.13 threadpoolctl 2.1.0 tifffile 2020.10.1 toml 0.10.2 toolz 0.11.1 torch 1.4.0 torchvision 0.5.0 tornado 6.0.2 tqdm 4.31.1 traitlets 4.3.3 typed-ast 1.4.3 typer 0.3.1 typing_extensions 4.1.1 ujson 4.0.2 unicodecsv 0.14.1 urllib3 1.26.8 wasabi 0.8.2 watchdog 2.1.2 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 0.58.0 Werkzeug 2.0.3 wheel 0.36.2 widgetsnbextension 3.5.1 wrapt 1.11.1 wurlitzer 2.1.0 xgboost 1.5.0 xlrd 2.0.1 XlsxWriter 1.4.3 xlwt 1.3.0 yapf 0.31.0 yarl 1.6.3 zict 2.0.0 zipp 3.6.0 zope.event 4.5.0 zope.interface 5.4.0
Stack Trace:
DataSplitter_global_timestamp: Cold users not allowed DataSplitter_global_timestamp: Verifying data consistency... DataSplitter_global_timestamp: Verifying data consistency... Passed! DataSplitter_global_timestamp: DataReader: app_rec_engine Num items: 373 Num users: 27058 Train interactions 65674, density 6.51E-03 Validation interactions 7729, density 7.66E-04 Test interactions 7243, density 7.18E-04 DataSplitter_global_timestamp: DataSplitter_global_timestamp: Done. TopPopRecommender: URM Detected 4 (6.78 %) cold items. EvaluatorHoldout: Processed 100 (100.0%) in 0.07 sec. Users per second: 1520 UserKNNCFRecommender: URM Detected 4 (6.78 %) cold items. Unable to load Cython Compute_Similarity, reverting to Python Similarity column 100 (100.0%), 17928.97 column/sec. Elapsed time 0.01 sec EvaluatorHoldout: Processed 100 (100.0%) in 0.06 sec. Users per second: 1626 --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-13-69a944593d51> in <module> 3 4 ----> 5 predictions = reczilla_inference(model_save_dict, "app_recs") 6 7 train_best_model(predictions, "app_recs", model_save_dict["metric_name"], <ipython-input-10-233c241cf0dd> in reczilla_inference(model_save_dict, dataset_split_path) 51 feat_test = np.array([metafeatures[feat_name] for feat_name in selected_feats])[np.newaxis, :] 52 print(feat_test) ---> 53 print(model_save_dict["model"].predict(feat_test)) 54 preds = np.squeeze(model_save_dict["model"].predict(feat_test)) 55 alg_perf = [(alg_name, pred) for alg_name, pred in zip(selected_algs, preds)] ~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/sklearn/multioutput.py in predict(self, X) 465 else: 466 X_aug = np.hstack((X, previous_predictions)) --> 467 Y_pred_chain[:, chain_idx] = estimator.predict(X_aug) 468 469 inv_order = np.empty_like(self.order_) ~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/xgboost/sklearn.py in predict(self, X, output_margin, ntree_limit, validate_features, base_margin, iteration_range) 877 ) 878 iteration_range = self._get_iteration_range(iteration_range) --> 879 if self._can_use_inplace_predict(): 880 try: 881 predts = self.get_booster().inplace_predict( ~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/xgboost/sklearn.py in _can_use_inplace_predict(self) 811 predictor = self.get_params().get("predictor", None) 812 if ( --> 813 not self.enable_categorical 814 and predictor in ("auto", None) 815 and self.booster != "gblinear" AttributeError: 'XGBRegressor' object has no attribute 'enable_categorical'
I was able to fix this with an older version of XGBoost: pip install xgboost==1.4.2
pip install xgboost==1.4.2
Trying to run inference on a new dataset with the following installed packages:
Stack Trace: