mylinhthibodeau
commented
6 years ago Dear @vmichalowski,
Great job on homework 10 !! You went above the requirements of the assignment and you probably learned a lot :)
Peer review
General requirements
- Reproducibility: completed successfully. Your code is entirely reproducible for the first part (Scraping Wikipedia data) and appears tidy in the second part as well (Twitter API, which requires a key), so I believe your code is functional and reproducible.
- Security: yes, twitter API key protected.
- Practicality: yes, criteria respected (no excessive data queries).
- Reporting: excellent reporting of your process in the homework 10 README, with examples of challenges and how you resolved them. I like the reflections you made on the meaning and the interpretation of data from Twitter. You also report your process in your code and your code comments are very helpful to understand the purpose of your code.
Prompt chosen: Scraping and API
Scrape data on average temperature in Canada from Wikipedia
Successful data queries and analyses
- You used the library xml2 and function
read_html()to retrieve Wikipedia data - You used the average temperature data from Wikipedia, and cleaned the messy data of Community and Average temperature columns.
- You made a plot to illustrate the data obtained
R wrapped API with Twitter
Successful data queries and analyses
- Used the twitterR R wrapped API
- Many functions (e.g.
twListToDF(),mutate()) used to format and clean the data - Interesting and stimulating homework to read because of the example provided on Sentiment according to location (West vs East Coast)
Additional information (not peer review)
I literally think you did an amazing job and that you showed in so many ways that you are motivated and keen on take on challenges by trying new things, so this section is only for your own learning and is not part of the peer review process.
I struggled a lot myself with cleaning data, and after many hours, I found some potentially helpful tips that you might also be interested in.
For your data cleaning: I found that you can combine multiple functions of stringr to perform subsetting strings and "intra-function" piping which allows me to cut the length of my code in half. For example, in your case, you could try something like:
temp_tableCA_clean_m1 <- temp_tableCA %>%
head(n=9) %>%
# Below: replace all brackets and numbers of Community with nothing (empty)
mutate(Community1 = str_replace_all(Community, c("\\[|\\]|1|2|3|5|6|7|8|9"), "")) %>%
# Mutate a new column July_Avg_High, subset the July high temperatures, then trim and then change to numeric in the same mutate function
mutate(July_Avg_High = substr(July.Avg..high..C...F.., start = 21, stop = 24) %>%
str_trim(side = "both") %>%
as.numeric()) %>%
# Mutate a new column July_Avg_Low, subset the July low temperatures, then trim and then change to numeric in the same mutate function
mutate(July_Avg_Low = substr(July.Avg..low..C...F.., start = 21, stop = 24) %>%
str_trim(side = "both") %>%
as.numeric())
head(temp_tableCA_clean_m1)I loved your p2 plot ! I found this trick for visualizing a plot in a different way by rotating it with p2 + coord_flip() here, you can try it out to see if you like rotating plots like me :)
You did an awesome job with this assignment and went above the requirements !
I wish you goodluck for the rest of your studies !
Regards,
My Linh Thibodeau
peterwhitman
@vincenzocoia @gvdr @ksedivyhaley @JoeyBernhardt @mynamedaike @pgonzaleze @derekcho
Here's a link to my rendered hw10 md file.