search

amritbhanu / EDM591_Hyperparameter

Hyper Parameter Optimization for Education Mining
0 stars 1 forks source link

Preprocessing #13

Open amritbhanu opened 6 years ago

amritbhanu commented 6 years ago

Change data preprocessing function. Check missing value; convert categorical value to binary; normalization. Finally make one function with parameter as column names, etc. Missing value: 3 ways Convert : 2 ways Normalization: 3 ways Try all 18 ways for preprocessing.

amritbhanu commented 6 years ago

@YiqiaoXu-Joe : check my comments here. https://github.com/amritbhanu/EDM591_Hyperparameter/commit/3b93e670239351cfe4c3d03fd82dc6f0ac9a4714

amritbhanu commented 6 years ago

@YiqiaoXu-Joe what is this line doing? https://github.com/amritbhanu/EDM591_Hyperparameter/blob/master/src/Preprocessing.py#L32

and also are you sure, now if we dont provide the column names it would only pick categorical columns to get dummies? I think it will get dummies for all columns. https://github.com/amritbhanu/EDM591_Hyperparameter/blob/master/src/Preprocessing.py#L51

amritbhanu commented 6 years ago

joe, is all your preprocessed data generated?

YiqiaoXu-Joe commented 6 years ago

I've generated new dataset by different categorical to numerical methods.