Possible SummaryEncoder doc error

[glevv]

Expected Behavior

SummaryEncoder should return N*cat_features columns, where N is the number of quantiles used to describe each category, at least this is stated in the original paper section 2.1

A generalization of the quantile encoder is to compute several features corre- sponding to different quantiles per each categorical feature, instead of a single feature

Actual Behavior

Docs example states that SummaryEncoder returns 1*cat_features

from category_encoders import * import pandas as pd from sklearn.datasets import load_boston bunch = load_boston() y = bunch.target X = pd.DataFrame(bunch.data, columns=bunch.feature_names) enc = SummaryEncoder(cols=["CHAS", "RAD"], quantiles=[0.25, 0.5, 0.75]).fit(X, y) numeric_dataset = enc.transform(X) print(numeric_dataset.info()) <class 'pandas.core.frame.DataFrame'> RangeIndex: 506 entries, 0 to 505 Data columns (total 13 columns): CRIM 506 non-null float64 ZN 506 non-null float64 INDUS 506 non-null float64 CHAS 506 non-null float64 NOX 506 non-null float64 RM 506 non-null float64 AGE 506 non-null float64 DIS 506 non-null float64 RAD 506 non-null float64 TAX 506 non-null float64 PTRATIO 506 non-null float64 B 506 non-null float64 LSTAT 506 non-null float64 dtypes: float64(13) memory usage: 51.5 KB None where it should be something like this

from category_encoders import * import pandas as pd from sklearn.datasets import load_boston bunch = load_boston() y = bunch.target X = pd.DataFrame(bunch.data, columns=bunch.feature_names) enc = SummaryEncoder(cols=["CHAS", "RAD"], quantiles=[0.25, 0.5, 0.75]).fit(X, y) numeric_dataset = enc.transform(X) print(numeric_dataset.info()) <class 'pandas.core.frame.DataFrame'> RangeIndex: 506 entries, 0 to 505 Data columns (total 17 columns): CRIM 506 non-null float64 ZN 506 non-null float64 INDUS 506 non-null float64 CHAS_25 506 non-null float64 CHAS_50 506 non-null float64 CHAS_75 506 non-null float64 NOX 506 non-null float64 RM 506 non-null float64 AGE 506 non-null float64 DIS 506 non-null float64 RAD_25 506 non-null float64 RAD_50 506 non-null float64 RAD_75 506 non-null float64 TAX 506 non-null float64 PTRATIO 506 non-null float64 B 506 non-null float64 LSTAT 506 non-null float64 dtypes: float64(17) memory usage: 51.5 KB None

[PaulWestenthanner]

You're right. @cmougan this was probably a copy-paste error?

[cmougan]

Yes, it's a copy paste issue.

It returns:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 506 entries, 0 to 505
Data columns (total 19 columns):
 #   Column   Non-Null Count  Dtype  
---  ------   --------------  -----  
 0   CRIM     506 non-null    float64
 1   ZN       506 non-null    float64
 2   INDUS    506 non-null    float64
 3   CHAS     506 non-null    float64
 4   NOX      506 non-null    float64
 5   RM       506 non-null    float64
 6   AGE      506 non-null    float64
 7   DIS      506 non-null    float64
 8   RAD      506 non-null    float64
 9   TAX      506 non-null    float64
 10  PTRATIO  506 non-null    float64
 11  B        506 non-null    float64
 12  LSTAT    506 non-null    float64
 13  CHAS_25  506 non-null    float64
 14  RAD_25   506 non-null    float64
 15  CHAS_50  506 non-null    float64
 16  RAD_50   506 non-null    float64
 17  CHAS_75  506 non-null    float64
 18  RAD_75   506 non-null    float64

Currently you can't use Summary Encoder or Quantile Encoder because they are not yet released. While there is not a new update of category_encoders package you can use the implementation that we use on the original paper in pip install sktools

[cmougan]

@PaulWestenthanner maybe we could do a package release?

[PaulWestenthanner]

We definitely should release. Unfortunately I do not have the rights to do so...

[cmougan]

@PaulWestenthanner who does?

[wdm0006]

@PaulWestenthanner you should have rights. If you update the version in init.py and the changelog, then go into the releases page of github and draft a new release (tag it with the release number) then the github action should take care of the rest.

[PaulWestenthanner]

Ah, I didn't know that. Sorry that I postponed the release for so long. It worked like charm though. The new version is visible in PyPI. Thanks a lot @wdm0006 !