Salary Prediction of Data Professions

[Harshit-code-tech]

Pull Request for ML-Crate 💡

Issue Title: Salary Prediction of Data Professions

• Info about the related issue (Aim of the project) : The goal of this project is to develop and enhance a machine learning model for predicting the salaries of data professionals based on various factors such as experience, job role, and performance ratings.
• Name: Harshit Ghosh
• Email ID for further communication: harshitghosh7@gmail.com
• GitHub ID: Harshit Ghosh
• Idenitfy yourself: Social Summer Of Code Season 3 Contributor

Closes: #670

Describe the add-ons or changes you've made 📃

Give a clear description of what have you added or modifications made

• Data Preprocessing Enhancements: Added handling for outliers, advanced feature engineering, and improved methods for handling missing values.
• Model Development: Implemented multiple machine learning algorithms, performed hyperparameter tuning, and used cross-validation for robustness.
• Model Evaluation: Evaluated models using performance metrics such as MAE, MSE, RMSE, and R2. Added visualization plots for better understanding.
• Documentation: Updated the README file to reflect enhancements,

Type of change ☑️

What sort of change have you made:

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Code style update (formatting, local variables)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [x] This change requires a documentation update

How Has This Been Tested? ⚙️

The following steps were taken to test the enhancements and ensure the reliability and performance of the Salary Prediction model:

1. Unit Testing:

   • Developed and executed unit tests for individual functions and methods used in data preprocessing, feature engineering, and model training.
   • Verified that each function performs as expected, handling various edge cases and inputs correctly.

2. Integration Testing:

   • Conducted integration tests to ensure that different components of the project (data preprocessing, model training, and evaluation) work together seamlessly.
   • Tested the entire data pipeline from data loading to model training and prediction to identify any issues arising from component interactions.

3. Model Evaluation:

   • Evaluated the performance of different models using multiple metrics: Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and R-squared (R²).
   • Used cross-validation to ensure robustness and prevent overfitting, checking consistency in model performance across different data splits.
   • Compared the results to baseline performance and previous iterations to confirm improvements.

4. Hyperparameter Tuning:

   • Performed hyperparameter tuning using Grid Search to find the optimal settings for each model, ensuring improved accuracy and performance.
   • Tested different combinations of hyperparameters to identify the best model configurations.

5. Exploratory Data Analysis (EDA) Validation:

   • Generated and reviewed visualization plots to validate the findings from Exploratory Data Analysis (EDA).
   • Ensured that visualizations correctly represent the data and provide insights into feature distributions and relationships.

6. Web Application Testing:

   • Integrated the best-performing model into the existing Flask web application.
   • Conducted end-to-end testing of the web application to ensure that the prediction functionality works as expected with the new model.
   • Validated the front-end interface to confirm it accommodates the new input features and changes in the model.

7. Documentation Review:

   • Reviewed and updated the project documentation to reflect the enhancements and changes made.
   • Ensured that the README file includes clear instructions on running the enhanced model and understanding the results.

8. Code Review:

   • Performed a thorough self-review of the code, ensuring it follows the project's guidelines and best practices.
   • Commented the code adequately, especially in complex or non-obvious sections, to improve readability and maintainability.
   • Made corresponding changes to the documentation to reflect the new features and improvements.

Verification

• Local Testing: Ran the project locally to verify that all components function correctly.
• Peer Review: Conducted peer reviews to identify potential issues and receive feedback on the enhancements and documentation.

Checklist: ☑️

• [x] My code follows the guidelines of this project.
• [x] I have performed a self-review of my own code.
• [x] I have commented my code, particularly wherever it was hard to understand.
• [x] I have made corresponding changes to the documentation.
• [x] My changes generate no new warnings.
• [x] I have added things that prove my fix is effective or that my feature works.
• [x] Any dependent changes have been merged and published in downstream modules.

[github-actions[bot]]

Our team will soon review your PR. Thanks @Harshit-code-tech :)

[Harshit-code-tech]

changes made @abhisheks008

[abhisheks008]

The README(2).md file should be renamed as README.md. And the README.md file should be removed.

[Harshit-code-tech]

change made @abhisheks008

[abhisheks008]

It was visible in the first go, but right now it's not visible

[Harshit-code-tech]

@abhisheks008 i don't know why that happen but now i had removed that link.. as it was not working as i wanted it..so i made it simple..please review it

[abhisheks008]

Need to do a minor change in the Web App/README file. Attach the demo video inside the README file, just copy the video from your local system and paste that inside the README file.

[Harshit-code-tech]

@abhisheks008 please check