shivajena
commented
2 years ago Thank you @andytai7 for taking time to write a detailed review of our milestone 1 submission, really appreciate it. We as a team are learning a lot from this group project, along with the collaborative practices. However, we have difficulty understanding some of your observations which we would need clarifications or rather insist on closer observations from your side as well as ours to address the gaps: 1. Project proposal {reasoning}: What sort of EDA will you do? What types of plots? Why? Any hypothesizes?
Our project answers a prediction question: predicting giant pumpkin weights, which we have very explicitly stated in our project proposal in Readme file. We have spoken about our approach of data preparation for EDA, some preliminary as well as important EDA observations and the link to detailed EDA report. For eg., we have mentioned about distributions of some of the attributes which could be our potential features, along with input and output forms as per the rubrics indicators of milestone 1. Therefore, we request you to have a relook into this.
What about class balance? There could be an imbalance in the classes, in which you would have to under-sample or oversample. Which one will you utilize?
Reiterating, we are dealing with a prediction problem of a continuous variable - giant pumpkin weight. This is not a classification problem, and hence, we are not sure about what exactly did you want to convey on class balance.
What about missing data? How will you handle the missing data?
Missing data handling is very clearly mentioned in our EDA report, and just an indication of data preparation is given in the project proposal since milestone 1 was not a detailed deployment of model where we needed to explore on missing data strategy (which we have indeed done in milestone 2 as desired).
Why only use linear regression? Have you thought of using wrapper algorithms (boruta algorithm) for feature selection?
We appreciate this suggestion, and we will plan on implementing wrapper algorithms in the future as we learn it in 573 but we haven’t till now. As of submissions till now, we have covered the various types of regression models in our project. However, the number of features are only limited in our data set(4-5 features only), and therefore, feature selection algorithms may not be very relevant in our context. Rather, feature engineering could be the key but we are trying our best to come up with others such as polynomial features or any other interesting learnings from 573.
Will you do cross-validation?
Yes, and infact we have touched upon this as well as the method of using pipe operators along with hyperparameter optimisation in the predictive modelling section of our proposal. Request you to kindly have a relook.
What about metrics? That was not touched upon in the project proposal. A suggestion for metrics to determine the performance of your models is Area Under Curve (AUC). The Area Under the Curve (AUC) measures the ability of a classifier to distinguish between classes and is used as a summary of the ROC curve. Also, look into and SHAP (Shapley Additive exPlanations), which explains the direction of each variable compared to the outcome variable.
Again, we are answering a prediction question and ROC-AUC are classification metrics, not regression. For regression context, we have explicitly mentioned metrics such as R-square score and accuracy as our initial scoring metrics. Request you to have a relook.
5. Exploratory data analysis in a literate code document: VIZ Comments If figure captions are not provided, the plot should be clearly explained in the text. I would recommend using figure captions: missing legends and X and Y-axis labels
We have provided figure captions and highlighted important observations in the our EDA report. We can discuss more on the specifics if needed.
This needs a lot more work to flesh out why you are doing certain EDA and data visualization. In addition, these reasons should be answered and informed how to proceed in terms of methods (ex., data transformation, data cleansing).
For prediction problems where number of features are less, data transformation and cleaning are very important and that is why we have mentioned detailed observations on these aspects in the data summary part of our EDA report. Had it been an inferential question or a prediction problem with large number of features, that case would have been perfect to incorporate specific EDA along with explanations. But it is not applicable as we understand for our project given such limit amount of features. 5. Exploratory data analysis in a literate code document: REASONING Comments The rationale is acceptable, but I don't know what plots relate to what. The one thing we were thoroughly focussing on is, observing any association between pumpkin weight and the features which were mostly categorical and very few numerical. This is what we have explained in the EDA report.
Given all of the above comments, we request you to have a relook into our proposal and EDA and if possible, have a regrade on these sections.
andytai7
imtvwy
3. Project proposal: reasoning Comments What sort of EDA will you do? What types of plots? Why? Any hypothesizes? What about class balance? There could be an imbalance in the classes, in which you would have to under-sample or oversample. Which one will you utilize?
What about missing data? How will you handle the missing data?
Why only use linear regression? Have you thought of using wrapper algorithms (boruta algorithm) for feature selection?
Will you do cross-validation?
What about metrics? That was not touched upon in the project proposal. A suggestion for metrics to determine the performance of your models is Area Under Curve (AUC). The Area Under the Curve (AUC) measures the ability of a classifier to distinguish between classes and is used as a summary of the ROC curve. Also, look into and SHAP (Shapley Additive exPlanations), which explains the direction of each variable compared to the outcome variable.
5. Exploratory data analysis in a literate code document: VIZ
Comments If figure captions are not provided, the plot should be clearly explained in the text. I would recommend using figure captions: missing legends and X and Y-axis labels.
This needs a lot more work to flesh out why you are doing certain EDA and data visualization. In addition, these reasons should be answered and informed how to proceed in terms of methods (ex., data transformation, data cleansing).
5. Exploratory data analysis in a literate code document: REASONING Comments The rationale is acceptable, but I don't know what plots relate to what.