Broaden skills in PS notebooks

[aolney]

In making corrections for the 7/20 online class, I've noticed the PS notebooks get pretty repetitive in terms of their analytic approach.

Thinking back to the summer internship, I think this was because I was concerned that students had fallen behind at this point, and the best way to ensure core competencies was to give as much practice on the same material as possible.

Some skills that are overrepresented:

• Correlation matrices
• Heatmaps

Some skills that are underrepresented:

• Scatterplot matrices
• Variable transformations (of dependent); larger issue here of expanding coverage of normalization (for predictors)
• Handling NA

I also suspect we could do more detailed model evaluation to complement the aggregate metrics typically used, i.e. under what conditions is the model output more reliable, diagnosing and correcting the reason why:

• Class-level performance
• Regression performance in certain parts of the range

[aolney]

Update on this with 2021 interns

• Some interns are struggling to complete the notebooks as-is, and seem to need practice on correlations/heatmaps
• So this issue may be more about adaptation to a given user's skill level than a one-size-fits-all rewrite

Suggested path forward

• Identify core elements/core curriculum
• Identify optional content
• Use student solution speed/correctness/student model to decide
  • Whether to unlock optional content
  • What optional content to unlock
• Overall we'd like everyone to finish in the time allotted but also fill that time

[aolney]

Update on this based on AB feedback - we are going to look at clickstream data to understand learning troublespots.

Therefore this issue is superseded by https://github.com/memphis-iis/datawhys-tracker/issues/50