Open ghost opened 3 years ago
Have you tried installing the package directly from github?
pip install git+https://github.com/jlevy44/InteractionTransformer
I downloaded the script from your GitHub and imported it into my notebook. If I install it directly from the GitHub I get the following error:
PicklingError Traceback (most recent call last)
I believe this could be due to the version of XGBoost that has issues with pickling the model. I believe downgrading XGBoost to 1.0.0 works if my memory serves me or alternatively specifying "single-threaded" for the dask_scheduler option, which should not pickle XGBoost but will take longer to run.
I am using the 1.0.0 version of XGBoost. I solved the pickle error, basically a file was missing. However, after running it with (and without) direct installation of the package directly from GitHub, I run again into the first error (Shape of passed values is (39, 1), indices imply (39, 39))
Hmm.. this should not happen. Are you running a multiclass problem? Before updating the package, can you also try deleting the "dist" directory if you are cloning the repo? This can cause issues.
It is a binary problem. I implemented your suggestion and it still does not run.
I experimented a bit. The code runs without problems for regression problems, but the aforementioned error occurs when a user opts for a binary problem
I have the same issue as the above. Did you already find the solution to the problem @jlevy44 ?
I have also tried it with your PythonDemo (example), but I get the same error. My code is as follows:
Which gives me the following error:
ValueError Traceback (most recent call last) ~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/internals/managers.py in create_block_manager_from_blocks(blocks, axes) 1670 blocks = [ -> 1671 make_block(values=blocks[0], placement=slice(0, len(axes[0]))) 1672 ]
~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/internals/blocks.py in make_block(values, placement, klass, ndim, dtype) 2743 -> 2744 return klass(values, ndim=ndim, placement=placement) 2745
~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/internals/blocks.py in init(self, values, placement, ndim) 130 raise ValueError( --> 131 f"Wrong number of items passed {len(self.values)}, " 132 f"placement implies {len(self.mgr_locs)}"
ValueError: Wrong number of items passed 1, placement implies 55
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
Hi all, were you able to work through these issues? Forking the repo and testing? I can get to this soon if there is still need.
Dear jlevy44,
I wanted to use the InteractionTransformer in combination with the XGBClassifier. Following your demo on GitHub, I run:
from xgboost import XGBClassifier transformer=InteractionTransformer(untrained_model=XGBClassifier(random_state=42, tree_method='hist'),max_train_test_samples=1000,mode_interaction_extract=int(np.sqrt(X_train.shape[1]))) transformer.fit(X_train,y_train)
Where my X_train and y_train are dataframes with shape (700000,39) and (700000,1), respectively.
I get the following error: --------------------------------------------------------------------------------- ValueError Traceback (most recent call last) C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\managers.py in create_block_manager_from_blocks(blocks, axes) 1661 blocks = [ -> 1662 make_block(values=blocks[0], placement=slice(0, len(axes[0]))) 1663 ]
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\blocks.py in make_block(values, placement, klass, ndim, dtype) 2721 -> 2722 return klass(values, ndim=ndim, placement=placement) 2723
C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals\blocks.py in init(self, values, placement, ndim) 129 if self._validate_ndim and self.ndim and len(self.mgr_locs) != len(self.values): --> 130 raise ValueError( 131 f"Wrong number of items passed {len(self.values)}, "
ValueError: Wrong number of items passed 1, placement implies 39
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)