[Review][Turing Data Story] "Who's protected by the first Covid19 lockdown?"

[crangelsmith]

Story Review:

Story Name: "Who's protected by the first Covid19 lockdown?"

Submitting Author: @DavidBeavan @crangelsmith @samvanstroud @kevinxufs

Pull Request: #79

Issue: #78

Reviewers: @nbarlowATI @LouiseABowler

Reviewer instructions & questions

@nbarlowATI, @LouiseABowler, please carry out your review in this issue by updating the checklist below, and writing new comments in case you have any questions. If you cannot edit the checklist please:

• Make sure you're logged in to your GitHub account
• Be sure to accept the invite at this URL: https://github.com/alan-turing-institute/TuringDataStories/settings/access

Any questions, concerns or suggestions regarding the review process please let @crangelsmith, @DavidBeavan or @samvanstroud know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review Checklist

Code of conduct

1. I confirm that I read and will adhere to the Turing Data Stories code of conduct.

General checks

2.1 Notebook: Is the source code for this data story available as a notebook in the linked pull request? 2.2 Contribution and authorship: Are the authors clearly listed? Does the author list seem appropriate and complete? 2.3 Scope and eligibility: Does the submission contain an original and complete analysis of open data? Is the story aligned with the Turing Data Stories vision statment?

Reproducibility

3.1 Does the notebook run in a local environment? 3.2 Does the notebook build and run in binder? 3.3 Are all data sources openly accessible and properly cited with a link? 3.4 Is the data provenance documented and transparent?

Pedagogy

4.1 Does the story demonstrate some specific data analysis or visualisation techniques? 4.2 Are these techniques well motivated? 4.3 Are these techniques well implemented? 4.4 Is the notebook well documented, using both markdown cells and comments in code cells? 4.5 Does the notebook has a introduction section motivating the story? 4.6 Does the notebook has a conclusion section discussing the main insight from the stories? 4.7 Is the paper well written (it does not require editing for structure, language, or writing quality)?

Context

5.1 Does the story give an insight into some societal issue? 5.2 Is the context around this issue well referenced (newspaper articles, scientific papers, etc.)?

AOB

Please note that there are some minor changes to occur on the notebook:

• [x] Add authorship (what each person has contributed, including reviewers)
• [x] Comment about NaNs in the rate factor dataframe
• [x] Change title to "first lockdown"

Updates

• @nbarlowATI has done his review and approved the story for publication
• AOB has been implemented in d7ceea4

[nbarlowATI]

Hi all! Thanks for asking me to review this Story - I am very impressed at first glance, and looking forward to a more detailed look.
I have a couple of comments on the review process though, that might be helpful:

• Would it be better to have two sets of checkboxes in this Issue, one for each of the reviewers? Otherwise it's not obvious who should tick them.
• It looks like the Binder buttons always build the master branch, so it's not really possible to review this branch on Binder. Not sure what the best solution is to this since it's a PR to go into master, so we will want to point to master eventually... Perhaps the binder button in each branch should point to that branch, but then after merging, make a separate commit to change the button to master?

[crangelsmith]

Hi @nbarlowATI thanks for your comments! Here some answers to your questions.

1. Good point! I think for this review you can just use the checkbox as a guideline of how to do this review. If there is one box point that is not working you can just refer to it as a comment to this issue. We changed the check box to a numbered list so is easy to refer to it (e.g 3.2 is not working).

2. Thanks we are implementing the binder point.

[DavidBeavan]

Update: The binder link from README.md, if you're in story/1 branch, should work to launch that story

[nbarlowATI]

Excellent, many thanks for the updates!
Here's my review:

1) I can confirm I've read the Code of Conduct and will adhere to it. 2.1) yes, notebook is available in PR. 2.2) author list not obvious on notebook, but I see this is mentioned in the AOB. 2.3) yes, this is a relevant and interesting study using open data. 3.1) yes, the notebook worked perfectly in my local environment. 3.2) yes, the notebook worked perfectly in Binder. (When clicking on the "Binder" button in this branch, Binder launched directly into this notebook. Presumably after the PR is merged into "master", the desired behaviour will be to to to the "stories" directory so the user can click on whatever notebook they want to see). 3.3) yes, all data sources are accessible and linked-to. 3.4) yes, the data provenance is documented and transparent. 4.1) yes. The analysis itself doesn't use cutting-edge techniques but does a very nice job of demonstrating some of the "bread-and-butter" data science building blocks - data wrangling, combining different data sources, looking at correlations, and making simple but clear visualizations. 4.2) yes, the techniques are well motivated. 4.3) yes, the techniques are well implemented. 4.4) yes, the notebook is well documented and explained. 4.5) yes, the notebook has an introductory section motivating the story. 4.6) yes, the notebook has a conclusions section discussing what has been learned. 4.7) yes, the notebook is very well written. 5.1) yes, the story gives insight into a highly relevant societal issue. 5.2) yes, I feel that the references and context for this study are sufficient, given that this particular topic is anyway extremely prominent in the literature and public consciousness.

[crangelsmith]

Hi @LouiseABowler! Do you have an idea when will you be able to submit your review? (No pressure, its just to get an idea :-) )

[LouiseABowler]

All done now! Here's the checklist - all the points are covered, I've put a couple of comments here but most are in the PR itself (#79).

Code of conduct

• [x] 1 I confirm that I read and will adhere to the Turing Data Stories code of conduct.

General checks

• [x] 2.1 Notebook: Is the source code for this data story available as a notebook in the linked pull request?
• [x] 2.2 Contribution and authorship: Are the authors clearly listed? Does the author list seem appropriate and complete?
• [x] 2.3 Scope and eligibility: Does the submission contain an original and complete analysis of open data? Is the story aligned with the Turing Data Stories vision statment?

Reproducibility

• [x] 3.1 Does the notebook run in a local environment?
  • I got it to run after I set up a conda environment based on binder/environment.yml, but the location of this file may not be immediately obvious to someone who hasn’t used Binder before. Could you add a short setup guide to the README? Some instructions for people who use Python without conda might also be useful.
• [x] 3.2 Does the notebook build and run in binder?
  • Yes, but the link in the README doesn’t work - the name of the notebook file has been changed so the filepath argument isn’t correct at the moment. I’ll make a suggestion in the review to fix :slightly_smiling_face:
  • The Binder also takes a long time to load, even after the image is prebuilt… I’ll investigate a bit more after I’ve finished the rest of this review.
• [x] 3.3 Are all data sources openly accessible and properly cited with a link?
• [x] 3.4 Is the data provenance documented and transparent?
  • I left one note to ask for some extra clarification where links directly to the files were used - you can see the source if you hover over the link to check the URLs, but it could be clearer.

Pedagogy

• [x] 4.1 Does the story demonstrate some specific data analysis or visualisation techniques?
• [x] 4.2 Are these techniques well motivated?
• [x] 4.3 Are these techniques well implemented?
• [x] 4.4 Is the notebook well documented, using both markdown cells and comments in code cells?
• [x] 4.5 Does the notebook has a introduction section motivating the story?
• [x] 4.6 Does the notebook has a conclusion section discussing the main insight from the stories?
• [x] 4.7 Is the paper well written (it does not require editing for structure, language, or writing quality)?
  • Yes, it reads really well! :rocket: I’ve left a couple of suggestions in the review, but all are minor changes.

Context

• [x] 5.1 Does the story give an insight into some societal issue?
• [x] 5.2 Is the context around this issue well referenced (newspaper articles, scientific papers, etc.)?

[samvanstroud]

Thank you @LouiseABowler, all your excellent suggestions have been implemented! I'm going to close this review and merge the story.