Titanic Survivor Predictor: A multiple machine learning model project to forecast survival outcomes of Titanic passengers. Engineered from historical data, refined with feature selection, tested with CI/CD, and outputs validation metrics.
To extend existing hand-rolled decision tree classifier into a RandomForestClassifier (RFC) by incorporating bootstrap aggregating (bagging) and feature randomness. This will improve prediction accuracy and control overfitting.
Tasks
[x] Bootstrap Sample Generation
Implement a function to generate bootstrap samples of the dataset.
Ensure each sample is the same size as the original dataset and is drawn with replacement.
[x] Feature Randomness in Splits
Modify the find_best_split function to consider a random subset of features when determining the best split.
Introduce a parameter to control the number of features to consider for each split.
[x] Build Trees with Random Features
Update the build_tree function to use the modified find_best_split that accounts for feature randomness.
Ensure that the tree construction process utilizes bootstrap samples and the random feature subsets.
[x] RandomForestClassifier Implementation
Implement the RandomForestClassifier class with methods for fitting the model and making predictions.
Include parameters for specifying the number of trees (n_trees) and optionally the number of features to consider at each split (n_features).
[x] Aggregating Predictions
For classification, implement a method that aggregates predictions from all trees in the forest by majority voting.
Ensure the predict method works for both single data points and batches of data points. (see user_passenger.py as possible example of taking single data point)
[ ] Testing
Write unit tests for each component (bootstrap sampling, feature selection, tree building, and prediction).
[ ] Performance Optimization (Stretch)
Profile the code to identify bottlenecks.
Optimize the implementation for speed and memory usage, if necessary.
Logic and Considerations
Bootstrap Sampling: This introduces randomness in the data each tree sees, reducing overfitting by averaging out biases.
Feature Randomness: Selecting a random subset of features for each split adds diversity to the trees, making the forest more robust.
Number of Trees: More trees can reduce variance but at the cost of increased computational complexity.
Majority Voting: This is a simple yet effective way to combine the predictions from all the trees in the forest.
Objective
To extend existing hand-rolled decision tree classifier into a RandomForestClassifier (RFC) by incorporating bootstrap aggregating (bagging) and feature randomness. This will improve prediction accuracy and control overfitting.
Tasks
[x] Bootstrap Sample Generation
[x] Feature Randomness in Splits
find_best_splitfunction to consider a random subset of features when determining the best split.[x] Build Trees with Random Features
build_treefunction to use the modifiedfind_best_splitthat accounts for feature randomness.[x] RandomForestClassifier Implementation
RandomForestClassifierclass with methods for fitting the model and making predictions.n_trees) and optionally the number of features to consider at each split (n_features).[x] Aggregating Predictions
[ ] Testing
[ ] Performance Optimization (Stretch)
Logic and Considerations
Testing