Ensures that the input df is a pandas DataFrame. This is crucial because the function performs operations specific to pandas DataFrames.
if not isinstance(df, pd.DataFrame):
raise TypeError("data_preprocessing_core(): The 'df' parameter must be a pandas DataFrame.")
List and String Type Checks for Columns
Verifies that x_cols is a list of strings, as these are expected to be column names in the DataFrame. It also checks that y_col is a string since it specifies the target column.
if not isinstance(x_cols, list) or not all(isinstance(col, str) for col in x_cols):
raise TypeError("data_preprocessing_core(): The 'x_cols' parameter must be a list of strings representing column names.")
if not isinstance(y_col, str):
raise TypeError("data_preprocessing_core(): The 'y_col' parameter must be a string representing the target column name.")
String Type Check for Data State
Ensures data_state is a string. This parameter indicates whether the data is already preprocessed or needs preprocessing.
if not isinstance(data_state, str):
raise TypeError("data_preprocessing_core(): The 'data_state' parameter must be a string.")
Float and Integer Type Checks
Checks the types for test_size and random_state to ensure they are a float and an integer, respectively. This is important for correctly setting up the data split configuration.
if not isinstance(test_size, float):
raise TypeError("data_preprocessing_core(): The 'test_size' parameter must be a float.")
if not isinstance(random_state, int):
raise TypeError("data_preprocessing_core(): The 'random_state' parameter must be an integer.")
Integer Type Check for Verbose
Confirms that verbose is an integer, which is used to control the level of output detail.
if not isinstance(verbose, int):
raise TypeError("data_preprocessing_core(): 'verbose' must be an integer value.")
Validation for Transformers and Vectorizers
Ensures that all transformers and vectorizers have the required methods (fit_transform, fit, and transform) to be used in the preprocessing pipeline.
if not (hasattr(numeric_imputer, 'fit_transform') or (hasattr(numeric_imputer, 'fit') and hasattr(numeric_imputer, 'transform'))):
raise TypeError("data_preprocessing_core(): The 'numeric_imputer' must support 'fit_transform' or both 'fit' and 'transform' methods.")
# Similar checks for other transformers and vectorizers...
Value Validations
Data Presence Check
Ensures that the DataFrame is not empty to guarantee that there is data available for preprocessing.
if df.empty:
raise ValueError("data_preprocessing_core(): The input DataFrame is empty.")
Data State Validation
Confirms that data_state matches either 'unprocessed' or 'preprocessed'. This is crucial for determining whether to apply preprocessing steps.
if data_state.lower() not in ['unprocessed', 'preprocessed']:
raise ValueError("data_preprocessing_core(): The 'data_state' parameter must be either 'unprocessed' or 'preprocessed'.")
Test Size Range Check
Ensures test_size is between 0 and 1 to correctly partition the dataset into training and testing sets.
if not 0 < test_size < 1:
raise ValueError("data_preprocessing_core(): The 'test_size' parameter must be a float between 0 and 1.")
Column Presence Check
Verifies that y_col and all x_cols are present in the DataFrame. This is essential for selecting the right data for preprocessing.
if y_col not in df.columns:
raise ValueError(f"data_preprocessing_core(): The specified target column '{y_col}' is not present in the DataFrame.")
missing_cols = set(x_cols) - set(df.columns)
if missing_cols:
raise ValueError(f"data_preprocessing_core(): The following feature columns are not present in the DataFrame: {', '.join(missing_cols)}")
Error Handling in
data_preprocessing_core()
Type Validations
df
is a pandas DataFrame. This is crucial because the function performs operations specific to pandas DataFrames.x_cols
is a list of strings, as these are expected to be column names in the DataFrame. It also checks thaty_col
is a string since it specifies the target column.data_state
is a string. This parameter indicates whether the data is already preprocessed or needs preprocessing.test_size
andrandom_state
to ensure they are a float and an integer, respectively. This is important for correctly setting up the data split configuration.verbose
is an integer, which is used to control the level of output detail.fit_transform
,fit
, andtransform
) to be used in the preprocessing pipeline.Value Validations
data_state
matches either 'unprocessed' or 'preprocessed'. This is crucial for determining whether to apply preprocessing steps.test_size
is between 0 and 1 to correctly partition the dataset into training and testing sets.y_col
and allx_cols
are present in the DataFrame. This is essential for selecting the right data for preprocessing.