Comments on the Report - Githubissues

Hi Eric,

As requested, I took a look at your project. Rather than grading based on the rubric, I'll focus on the report itself.

Let me start with what I liked:

You clearly explained your thought process, and I loved seeing the transition from the initial hypothesis to the framing of the classification problem.
You included "setbacks" (approaches that did not work as well as you intended). In data science, other people may be interested in building on top of your work, and highlighting the approaches that don't work is very helpful.
You discussed the time it took for different models. In many applications that are making predictions in real time, computation time is a constraint.

Constructive feedback:

When calculating computation time for models, calculate both the time it takes to fit a model and the time it takes to generate predictions. Maybe a model takes longer overall, but what if it takes significantly less time to generate predictions?
I would have liked to see even more exploratory data analysis. Although you don't have to report on everything you did for EDA, you should include just enough to show that you have an understanding of the dataset. Specifically, for each variable in the dataset, you should know the distribution, the number of missing values, and outliers. Also, you should understand how variables interact with one another. In the real world, it would also be essential to know how that data was collected, and if it's human input, what groups were targetted. All this is admittedly a lot of work, and I have even heard of data science teams doing "Exploratory Data Analysis Parties." However, this is a key differentiator between average data scientists and great ones.
It's not clear to me what "score" is. For regression, are you using MAE or MSE? For classification, are you looking at accuracy, precision/recall, log-loss score, or etc? Given that I know the output from Sklearn score method for classification is accuracy, should you also look at precision/recall? What are the dangers of omitting that (think imbalanced classes)?
Speaking of class imbalance, is it dangerous to use imputation by median if there is a class imbalance?
Why did you conclude that your classification models were overfitting? Try creating an addition split, so you have training, validation, and test sets, and you would use validation score to choose between models and only evaluate the test score at the very end. There are actually people pushing for even more splitting and creating a separate exploratory set
Your python code looks great, but I think it would've been even better in a notebook file. Within a notebook file, you can directly embed plots and include your thoughts in markdown. Most other data science people I know prefer reading code on notebooks, and it will give you more opportunity to share your work with potential employers.

I know I included a lot of constructive feedback items, and there will always be room for improving projects. However, you should be proud of what you've accomplished, and you definitely stretched yourself. For example, I know that imputation was not covered in the course, and it was very cool to see you implement it.

I would recommend shelving this particular project for a few weeks and to come back to it with a fresh mind. In the meantime, try out a few challenges on Kaggle and participate in discussions there to further improve your skills. Feel free to loop me in if you want me to take a look at your future DS/ML projects.

SLOEric / date-a-scientist

Comments on the Report #3