Progress Assessment & A/B Testing for Lessons

[waaaaargh]

In a meatspace-based discussion yesterday, this came up. I assigned @CruzR because essentially, it was his idea.

Acceptance Criteria

• A process is in place for defining tests that enable learners to assess their process independent of projects.
• A process is in place for associating project-independent progress assessments with fitting projects

[CruzR]

Just writing down some thoughts I had on this:

• Have a set of quiz questions for each learning goal
• Randomly partition this set into two subsets, with about 3-5 minutes worth of questions in each
• Pose the first set of questions before a lesson/project, the second set of questions afterwards

This way, we can assess whether a learner actually has profitted from a lesson/project or whether they just already knew the stuff being taught. The random partitioning prevents us from accidentally putting harder questions in one of the subsets. If we also want to assess long-term retainment of the learnt material, we can simply partition into three subsets and let participants answer the last subset a couple of weeks later.