search

py-why / causaltune

AutoML for causal inference.
Apache License 2.0
202 stars 29 forks source link

Evaluation failed error for Quick Start Example #297

Closed patriciowoodley closed 2 months ago

patriciowoodley commented 11 months ago

Describe the bug A clear and concise description of what the bug is.

I've tried to reproduce the following code:

from causaltune import CausalTune
from causaltune.datasets import synth_ihdp
# prepare dataset
data = synth_ihdp()
data.preprocess_dataset()
# init CausalTune object with chosen metric to optimise
ct = CausalTune(time_budget=600, metric="energy_distance")
# run CausalTune
ct.fit(data)
# return best estimator
print(f"Best estimator: {ct.best_estimator}")

and I got the following error:

Fitting a Propensity-Weighted scoring estimator to be used in scoring tasks Initial configs: [{'estimator': {'estimator_name': 'backdoor.causaltune.models.NaiveDummy'}}, {'estimator': {'estimator_name': 'backdoor.causaltune.models.Dummy'}}, {'estimator': {'estimator_name': 'backdoor.econml.metalearners.SLearner'}}, {'estimator': {'estimator_name': 'backdoor.econml.metalearners.DomainAdaptationLearner'}}, {'estimator': {'estimator_name': 'backdoor.econml.dr.ForestDRLearner', 'min_propensity': 1e-06, 'n_estimators': 100, 'min_samples_split': 5, 'min_samples_leaf': 5, 'min_weight_fraction_leaf': 0.0, 'max_features': 'auto', 'min_impurity_decrease': 0.0, 'max_samples': 0.45, 'min_balancedness_tol': 0.45, 'honest': True, 'subforest_size': 4}}, {'estimator': {'estimator_name': 'backdoor.econml.dml.CausalForestDML', 'drate': True, 'n_estimators': 100, 'criterion': 'mse', 'min_samples_split': 10, 'min_samples_leaf': 5, 'min_weight_fraction_leaf': 0.0, 'max_features': 'auto', 'min_impurity_decrease': 0.0, 'max_samples': 0.45, 'min_balancedness_tol': 0.45, 'honest': True, 'fit_intercept': True, 'subforest_size': 4}}, {'estimator': {'estimator_name': 'backdoor.causaltune.models.TransformedOutcome'}}] Evaluation failed! {} Traceback (most recent call last): File "/home/jovyan/causaltune/causaltune/optimiser.py", line 579, in _estimate_effect estimate = self._est_effect_stub(method_params) File "/home/jovyan/causaltune/causaltune/optimiser.py", line 553, in _est_effect_stub return self.causal_model.estimate_effect( File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/dowhy/causal_model.py", line 318, in estimate_effect return estimate_effect( File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/dowhy/causal_estimator.py", line 711, in estimate_effect estimator.fit( File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/dowhy/causal_estimators/econml.py", line 200, in fit self.estimator.fit(estimator_data_args, kwargs) File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/econml/_cate_estimator.py", line 130, in call m(self, Y, T, *args, *kwargs) File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/econml/metalearners/_metalearners.py", line 222, in fit self.overall_model.fit(feat_arr, Y) File "/home/jovyan/causaltune/causaltune/models/monkey_patches.py", line 124, in fit super().fit(X_train, self._preprocess_y(y_train), args, **kwargs) File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/flaml/automl.py", line 2872, in fit self._state.time_budget = time_budget if time_budget > 0 else 1e10 TypeError: '>' not supported between instances of 'NoneType' and 'int'

Evaluation failed! {} Traceback (most recent call last): File "/home/jovyan/causaltune/causaltune/optimiser.py", line 579, in _estimate_effect estimate = self._est_effect_stub(method_params) File "/home/jovyan/causaltune/causaltune/optimiser.py", line 553, in _est_effect_stub return self.causal_model.estimate_effect( File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/dowhy/causal_model.py", line 318, in estimate_effect return estimate_effect( File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/dowhy/causal_estimator.py", line 711, in estimate_effect estimator.fit( File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/dowhy/causal_estimators/econml.py", line 200, in fit self.estimator.fit(estimator_data_args, kwargs) File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/econml/_cate_estimator.py", line 130, in call m(self, Y, T, *args, *kwargs) File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/econml/metalearners/_metalearners.py", line 521, in fit self._fit_weighted_pipeline(self.models_control[ind], X[T == 0], Y[T == 0], File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/econml/metalearners/_metalearners.py", line 560, in _fit_weighted_pipeline model_instance.fit(X, y, sample_weight) File "/home/jovyan/causaltune/causaltune/models/monkey_patches.py", line 124, in fit super().fit(X_train, self._preprocess_y(y_train), args, **kwargs) File "/home/jovyan/my_conda_envs/causaltune_source/lib/python3.9/site-packages/flaml/automl.py", line 2872, in fit self._state.time_budget = time_budget if time_budget > 0 else 1e10 TypeError: '>' not supported between instances of 'NoneType' and 'int'

Your environment

Anaconda environment, installing causaltune from Source

packages in environment at /home/jovyan/my_conda_envs/causaltune_source:

#

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge alabaster 0.7.13 pypi_0 pypi asttokens 2.4.1 pypi_0 pypi babel 2.13.1 pypi_0 pypi bzip2 1.0.8 hd590300_5 conda-forge ca-certificates 2023.11.17 hbcca054_0 conda-forge causal-learn 0.1.3.7 pypi_0 pypi causaltune 0.1.3 pypi_0 pypi certifi 2023.11.17 pypi_0 pypi charset-normalizer 3.3.2 pypi_0 pypi cloudpickle 3.0.0 pypi_0 pypi comm 0.2.0 pypi_0 pypi contourpy 1.2.0 pypi_0 pypi cycler 0.12.1 pypi_0 pypi cython 0.29.36 pypi_0 pypi dcor 0.6 pypi_0 pypi debugpy 1.8.0 pypi_0 pypi decorator 5.1.1 pypi_0 pypi docutils 0.19 pypi_0 pypi dowhy 0.9.1 pypi_0 pypi econml 0.14.1 pypi_0 pypi exceptiongroup 1.2.0 pypi_0 pypi executing 2.0.1 pypi_0 pypi flaml 1.0.14 pypi_0 pypi fonttools 4.45.1 pypi_0 pypi holidays 0.37 pypi_0 pypi idna 3.6 pypi_0 pypi imagesize 1.4.1 pypi_0 pypi importlib-metadata 6.8.0 pypi_0 pypi importlib-resources 6.1.1 pypi_0 pypi iniconfig 2.0.0 pypi_0 pypi ipykernel 6.27.1 pypi_0 pypi ipython 8.18.1 pypi_0 pypi jedi 0.19.1 pypi_0 pypi jinja2 3.1.2 pypi_0 pypi joblib 1.3.2 pypi_0 pypi jupyter-client 8.6.0 pypi_0 pypi jupyter-core 5.5.0 pypi_0 pypi kaleido 0.2.1 pypi_0 pypi kiwisolver 1.4.5 pypi_0 pypi ld_impl_linux-64 2.40 h41732ed_0 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 13.2.0 h807b86a_3 conda-forge libgomp 13.2.0 h807b86a_3 conda-forge libnsl 2.0.1 hd590300_0 conda-forge libsqlite 3.44.2 h2797004_0 conda-forge libuuid 2.38.1 h0b41bf4_0 conda-forge libzlib 1.2.13 hd590300_5 conda-forge lightgbm 4.1.0 pypi_0 pypi llvmlite 0.41.1 pypi_0 pypi markupsafe 2.1.3 pypi_0 pypi matplotlib 3.8.2 pypi_0 pypi matplotlib-inline 0.1.6 pypi_0 pypi mpmath 1.3.0 pypi_0 pypi ncurses 6.4 h59595ed_2 conda-forge nest-asyncio 1.5.8 pypi_0 pypi networkx 2.8.8 pypi_0 pypi numba 0.58.1 pypi_0 pypi numpy 1.23.5 pypi_0 pypi openssl 3.2.0 hd590300_1 conda-forge packaging 23.2 pypi_0 pypi pandas 1.5.3 pypi_0 pypi parso 0.8.3 pypi_0 pypi patsy 0.5.3 pypi_0 pypi pexpect 4.9.0 pypi_0 pypi pillow 10.1.0 pypi_0 pypi pip 23.3.1 pyhd8ed1ab_0 conda-forge pivottablejs 0.9.0 pypi_0 pypi platformdirs 4.0.0 pypi_0 pypi plotly 5.18.0 pypi_0 pypi pluggy 1.3.0 pypi_0 pypi prompt-toolkit 3.0.41 pypi_0 pypi psutil 5.9.6 pypi_0 pypi ptyprocess 0.7.0 pypi_0 pypi pure-eval 0.2.2 pypi_0 pypi pydot 1.4.2 pypi_0 pypi pygments 2.17.2 pypi_0 pypi pyparsing 3.1.1 pypi_0 pypi pytest 7.4.3 pypi_0 pypi python 3.9.18 h0755675_0_cpython conda-forge python-dateutil 2.8.2 pypi_0 pypi python-graphviz 0.20.1 pypi_0 pypi pytz 2023.3.post1 pypi_0 pypi pyzmq 25.1.1 pypi_0 pypi readline 8.2 h8228510_1 conda-forge requests 2.31.0 pypi_0 pypi scikit-learn 1.1.3 pypi_0 pypi scipy 1.11.4 pypi_0 pypi seaborn 0.13.0 pypi_0 pypi setuptools 65.5.1 pypi_0 pypi shap 0.41.0 pypi_0 pypi six 1.16.0 pypi_0 pypi slicer 0.0.7 pypi_0 pypi snowballstemmer 2.2.0 pypi_0 pypi sparse 0.14.0 pypi_0 pypi sphinx 5.3.0 pypi_0 pypi sphinx-design 0.3.0 pypi_0 pypi sphinxcontrib-applehelp 1.0.7 pypi_0 pypi sphinxcontrib-devhelp 1.0.5 pypi_0 pypi sphinxcontrib-htmlhelp 2.0.4 pypi_0 pypi sphinxcontrib-jsmath 1.0.1 pypi_0 pypi sphinxcontrib-qthelp 1.0.6 pypi_0 pypi sphinxcontrib-serializinghtml 1.1.9 pypi_0 pypi stack-data 0.6.3 pypi_0 pypi statsmodels 0.13.5 pypi_0 pypi sympy 1.12 pypi_0 pypi tenacity 8.2.3 pypi_0 pypi threadpoolctl 3.2.0 pypi_0 pypi tk 8.6.13 noxft_h4845f30_101 conda-forge tomli 2.0.1 pypi_0 pypi tornado 6.4 pypi_0 pypi tqdm 4.66.1 pypi_0 pypi traitlets 5.14.0 pypi_0 pypi typing-extensions 4.8.0 pypi_0 pypi tzdata 2023c h71feb2d_0 conda-forge urllib3 2.1.0 pypi_0 pypi wcwidth 0.2.12 pypi_0 pypi wheel 0.42.0 pyhd8ed1ab_0 conda-forge wise-pizza 0.2.0 pypi_0 pypi xgboost 2.0.2 pypi_0 pypi xz 5.2.6 h166bdaf_0 conda-forge zipp 3.17.0 pypi_0 pypi

AlxdrPolyakov commented 2 months ago

Hello, please install new version causaltune==0.1.4 and try again!