WARNING: These fixes may change the behavior of DataFrameWithInfo class (attributes and methods) and its APIs
This a list of TODOs in DataFrameWithInfo class that need further inspection now that the package scope is more clear.
[ ] In column_list_by_type property exclude NaN columns (self.nan_cols) from col_list too (so they will not be included in num_categorical_cols just for one not-Nan value)
[ ] In column_list_by_type property, when returning ColumnListByType instance, numerical_cols should not includebool_cols
[ ] In to_be_encoded_cat_cols property, probably cols_by_type.num_categorical_cols (that are categorical columns with numerical values) should not need encoding and should not be included in categorical_cols returned variable.
[ ] In least_nan_cols method, add argument col_list to select the columns to analyze when the user wants to find which columns have the lowest count of NaNs
[ ] Rename check_duplicated_features method to contains_duplicated_features because it returns a boolean value
[ ] In check_duplicated_features method, in case there are columns with the same name, check if their values are the same too and inform the user appropriately.
[x] Think about the case where the same column is among both "original_columns" and "derived_columns". Does this make sense? Should we raise an error? Would we add the same FeatureOperation twice to the column? That could create problems when looking for that operation because two are found! This case is supposed to happen when a column provides the original values but it is also used to store the resulting values. Is this a good practice? (see the following checkbox)
[x] In get_enc_column_from_original, the found operation found_operat should never be also among the found_operat.derived_columns (according to the previous checkbox). This similar check should be removed in get_original_from_enc_column method.
[x] In add_operation method, the feature_operation argument should not contain original_columns or derived_columns = None because it means they are not specified (instead they are supposed to be set to ()). So the code to raise this error should be implemented.
List of TODOs in FeatureOperation class:
[x] Should this class be moved to another file along with the other DataFrameWithInfo supplementary functions?
[x] Remove details attribute since the value in docstring is the same as encoded_values_map
[x] Remove one between encoder and encoding_function attribute (or maybe just distinguish between the instance and the class type. I do not think this is necessary since the encoding function is simply the type of the encoder)
WARNING: These fixes may change the behavior of DataFrameWithInfo class (attributes and methods) and its APIs
This a list of TODOs in DataFrameWithInfo class that need further inspection now that the package scope is more clear.
column_list_by_typeproperty exclude NaN columns (self.nan_cols) fromcol_listtoo (so they will not be included in num_categorical_cols just for one not-Nan value)column_list_by_typeproperty, when returning ColumnListByType instance,numerical_colsshould not includebool_colsto_be_encoded_cat_colsproperty, probablycols_by_type.num_categorical_cols(that are categorical columns with numerical values) should not need encoding and should not be included incategorical_colsreturned variable.least_nan_colsmethod, add argumentcol_listto select the columns to analyze when the user wants to find which columns have the lowest count of NaNscheck_duplicated_featuresmethod tocontains_duplicated_featuresbecause it returns a boolean valuecheck_duplicated_featuresmethod, in case there are columns with the same name, check if their values are the same too and inform the user appropriately.get_enc_column_from_original, the found operationfound_operatshould never be also among thefound_operat.derived_columns(according to the previous checkbox). This similar check should be removed inget_original_from_enc_columnmethod.add_operationmethod, thefeature_operationargument should not containoriginal_columnsorderived_columns= None because it means they are not specified (instead they are supposed to be set to()). So the code to raise this error should be implemented.List of TODOs in FeatureOperation class:
detailsattribute since the value in docstring is the same as encoded_values_mapencoderandencoding_functionattribute (or maybe just distinguish between the instance and the class type. I do not think this is necessary since the encoding function is simply the type of the encoder)