Normative answers & organization of misconceptions

[ivogeorg]

Two things to think about and include in app design (possibly 2nd tier):

1. Normative and/or canonical answers should be part of the Question data.
2. Misconceptions for each question should be categorized and organized as additional data. This is a foundation for some very exciting downstream uses of the App data. Just scratching the surface:
   1. Such data is rare though not altogether non-existent.
   2. Such data is hard to collect but this App, which has the questions, the normative answers, and the student answers, subsumes any separate collection task.
   3. Such data is a treasure trove for Conceptual Change Research, based on NLP.

These are recorded as a TODO under Question design.

[JeffLoats]

This makes me think of a few design features we may want to build in. I don't think these are MVP.

1. A "Follow-up notes" field for each question, which students can see if they review an assignment after it is due.
2. An "Instructor notes" field, viewable by instructors in the full question view. This field, in particular, could be used for all kinds of information like is suggested here (common misconceptions, etc.). This could also be used as an optional change-log if modifying a question.
3. Maybe we want to build in another field or two for possible use later, even if they are hidden for now. I don't know how hard/easy it would be to later add 2 fields to every question in the system.

[ivogeorg]

These should be added to our Stories, probably in Design Spike 5, since 4 is getting a bit cluttered and needs cleanup or migration to 5.

One point about misconceptions specifically: I don't want that data to be contained in a generic free-form field like the "Instruction notes" you suggest (we can still have that, of course); instead, they need to be structured data in the sense that we have a canonical definition of the misconception (very important for my goals in the domain of the structure of knowledge and the concept of learnability based on the structural criteria for learning spaces), along with the raw expression(s) from the student answers. This way we don't have to parse them out; they are already there in their database record columns. Does it make sense? I can explain further if not.

[JeffLoats]

I think it does make sense, but my impression is that turning this into structured data is real intellectual work that would take time. I am not confident we can expect almost any instructors who use the system to circle back to questions after they are used and create this kind of structured data for us.

On Tue, Aug 24, 2021 at 4:15 PM Ivo Georgiev @.***> wrote:

These should be added to our Stories, probably in Design Spike 5, since 4 is getting a bit cluttered and needs cleanup or migration to 5.

One point about misconceptions specifically: I don't want that to be a generic field like the "Instruction notes" you suggest (we can still have that, of course); instead, they need to be structured data in the sense that we have a canonical definition of the misconception (very important for my goals in the domain of the structure of knowledge), along with the raw expression(s) from the student answers. Does it make sense? I can explain further if not.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ivogeorg/jitt/issues/2#issuecomment-905013260, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASRRKTR4IPRZERVQMDISG3TT6QKZVANCNFSM5CSGLICQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[ivogeorg]

I absolutely don't expect instructors to do a thing. We collect the data, anonymize it, get their consent to use it, and then get researchers (including me) to do the intellectual work. This would be quite an interesting and rare dataset.