Find problematic values in train and test - Githubissues

JoshVarty / KaggleUtils

A collection of utilities I use for EDA, feature engineering etc.

MIT License

1 stars 1 forks source link

Find problematic values in train and test #5

Open JoshVarty opened 5 years ago

JoshVarty commented 5 years ago

Automatically look for:

Constant values
- train.nunique(axis=1) == 1
- Values that occur as > 99.9% in a column
- Values that only occur in one set
- Look for duplicate rows
- with different target values.
- with the same target values

Maybe automatically run

train.T.drop_duplicates()

and

for f in categorical_features:
  train[f] = train[f].factorize()
traintest.T.drop_duplicates()