Turing Commons collaboration

[gmingas]

Regular meeting notes here

Initial conversation: We had a meeting with Chris from Turing Commons today (description of course here) and discussed several things:

• Turing Commons Syllabus: 3 independent modules (ethics & governance, responsible research (prob greatest overlap with RDS course), public communication). Course will include ~10 lectures (of ~1 hour each) + exercises. No pre-requisite to know Python. Should know general statistics/data analysis. Qualitative assessment (exercises to think through problems, not exercises with fixed solutions).
• They have a fairly large budget, including for graphic design.
• No requirement for single dataset used throughout the code - probably want something for exploratory data analysis, and something for reporting.
• Design workshops in June - we could go to observe if we like
• Overlaps with their course: Turing Commons will more focus on theoretical aspects of ethics in AI, going through different stages of the analytics life cycle, giving examples of types of biases and ethics issues that can arise in these stages. There will be exercises that will not involve coding for the most part but will look at some real/artificial datasets. Our course will focus more on the practical aspects when working with code and data, giving also some examples and touching some of the same issues as Turing Commons but not reaching the same depth. We will rather focus more on how these issues play out in practice during the hands-on sessions. The course are complementary.
• Collaboration: We can collaborate on sharing materials (e.g. harmonise lifecycle graphs and other charts, sharing exercises and datasets), where they could carry more weight on the theoretical ideas/contexts and we could carry more weight on exercises and practical scenarios.
• Datasets: This is under discussion. They are planning to use artificial data generated using rules-based parameterised models. These will allow them to introduce specific biases in the dataset as they please and talk about them, e.g. see this list and this resource. Their preference is to use healthcare, criminal justice or hiring data (in order of preference). Also look at this. We could collaborate on this and share data or go with a real dataset of our own.

[gmingas]

A fortnightly catch up has been set up for July/August (during the time the bulk of development will happen for both courses).

The following are potential topics of collaboration/discussion:

• Define interesting questions to ask based on the European Life Quality data and explore areas we want to highlight.
• Co-design some of the exercises and provide interesting visualisations and reproducible code that both courses could use during teaching
• Discuss how EDI and ethics of data science are discussed in the two courses and possible overlaps
• Harmonisation of some of the material, e.g. some figures and terminology

[gmingas]

RRP self assessment repo: https://github.com/alan-turing-institute/rrp-selfassessment

[gmingas]

Data Science life cycle:

[gmingas]

Feedback from TC sessions in June: Chris shared notes from the three workshops organised in June to get feedback on the three modules of the Turing Commons course. The second one is closest to what we want to do in the RDS course from an ethics perspective:

• https://hackmd.io/@cburr/ryIE9WRjO
• https://hackmd.io/@cburr/rJB5-XJnu
• https://hackmd.io/@cburr/r1sCbKJ2d

[gmingas]

Some ideas about overlapping content on biases:

• Turing Commons course wants to demonsstrate some possible biases in the data science life cycle through examples with real data. Specifically, the following are of particular interest according to Chris:
  • Pre-processing
    • Show off missing data biases, impact of different imputation methods
  • Model development: techniques to treat dataset imbalances and fairness, trade-offs when doing it.
  • Visualisation/explainability: Use novel packages to communicate results in an interesting way and support explainability. Splink demo notebooks: bayes intuition chart: https://github.com/moj-analytical-services/splink_demos
• Chris will share a hackmd document to asynchronously agree on overlapping material and take the above Turing Commons requirements into account when designing our modules. I think the following document has a lot of useful information on different biases that we could use anyway for our taught sessions even if they do not overlap with what Turing Commons is doing: https://alan-turing-institute.github.io/rrp-selfassessment/bias/bias-intro.html

[gmingas]

@chrisdburr Have you had a chance to sketch the hackmd mentioned above?

[chrisdburr]

Hi, @gmingas. I’ve drafted something, and am still making some changes to the text. However, feel free to start reading and commenting: HackMD Page

[gmingas]

See link below with our attempt to take the TRIPOD framework (first tab) and adapt it to the data science project lifecycle (including editing some items in the checklist and defining with which steps in the lifecycle these items overlap). The mapping is shown in the second tab. This is WIP but can be used by attendees as a checklist of things they should be asking themselves and documenting during the hands-on sessions. We plan to have a second session next week (organised by Callum) to finish this and probably add/edit items based on the scoping questions defined in lesson 1.2.

Spreadsheet