Prototype Quiz

[chloeatplay]

Prototype Quiz Content as described on ticket #12 cc @LucasBlair :question: :pencil2:

This could involve creating sketches, a google form and or paper prototypes. Cc @emgollie to include in her user testing plan #15 :pray:

[emgollie]

Added as early as DML. Is that too soon @ghami? Would love your and @LucasBlair's thoughts: https://openbadges.etherpad.mozilla.org/pathways-userstories

[chloeatplay]

First prototype for Quiz over here https://docs.google.com/forms/d/1sjikzL8PV-E9cIf-_Vv51zFcnpiWlkhsJKwqyiCWae4/viewform We will gather feedback at DML and provide suggestions for the next iteration :balloon: cc/ @ghami @emgollie

[threeqube]

Yay! :dancer:

[emgollie]

Thanks @chloeatplay @ghami! A few thoughts upon reviewing:

1. I think we can shorten the intro paragraph to get participants into the survey faster. I'll take a stab at shortened language in the AM.
2. I would re-phrase the first prompt "I want a career that mostly allows me to" to "The #1 most important thing to me in my future career is..."
3. On the second question, I would edit "I might like to do practical things" to "I work best..."
4. Can we clarify this prompt for the third question? "What context you might like to work in most from the following..."
5. For the fourth question, can we change "Tell us a bit about things you like to do best: I like..." to "...I MOST like"?
6. Same with the next question: people may be tempted to answer all the general statements and I think it's important to suggest that we're looking for the single response that fits best out of all that are available.
7. Might we want to phrase "Choose which of the following describe you..." to a question about how the people who know them perceive them? This might help get to the reliability question, etc., and help with soliciting more objective responses.
8. For the grit question, we should add "I am most likely to..." before "...do you give up or continue?" I would suggest writing each of these prompts in the first person to make them feel more personal.
9. On the prompt "Do you like the big picture or digging into the detail?" I think we need to better describe the 1 and the 5 on the Likert scale. I might say "'I like focusing on the big picture on a project (as opposed digging into detail).' To what extent does this describe how you usually perceive your work?"
10. Suggest phrasing "Use the scale to show where you might prefer working" to "The environment I most prefer working in is..." with the scale the way you have it.
11. For "Choose where you might like to work most if you were working outdoors..." should we add a park and an other: _____ write-in option?
12. The badges questions feel a bit general at this phase, but I think we can make a few easy tweaks to get to more specificity.
13. The overall survey may take a while given the various question types (which tend to slow people down). If we had to hold off on including a few of these for this round, which do we think we might forgo?

[chloeatplay]

Thanks @emgollie for comments, implemented in google form. For 13, I am going to rely on @ghami 's input.

[emgollie]

@ghami @threeqube: @chloeatplay and I worked on tighten this up slightly so we can send it to a few organizations (and their teens) next week. @ghami, we wondered if you could tell us about the question asked here? I wonder if we need a few other options as these don't appear to be either/or selections. We look forward to your thoughts--thanks!

[ghami]

The quiz prototype isn't really matching some of the functionality I had envisaged for the quiz as a lot of it is intended to enable choosing more than one answer. This would then be stacked behind the scenes to show highest level choice within the context of a range of choices. In this particular question though this intending to be a deeper dive based on a previous choice, with the intention being to try to start to refine choices to narrow down range of pathways to present as being relevant to that person. If that makes sense! Would actually be useful to talk through the logic I've been thinking question by question and section by section as it is quite challenging to get it across in things that don't have the functionality I'm hoping could be built in. Also need to check if the functionality can be built in!

[emgollie]

@ghami @chloeatplay Thanks very much for talking today! I did a bit more research into Holland codes and revised some of the question phrasing. It's good to know we can provide more response options, and I changed a few of the question formats to checkboxes vs. multiple choice. I do think we would want to combine a few of these questions to a single page so the quiz doesn't feel so lengthy.

I'm starting to wonder if a quiz taken remotely will give us the information we need, however. In trying to code the responses we get, I think it may be difficult to connect responses with potential future badges. Can we discuss tomorrow after the community call? I wonder if we make this a discussion guide for in-person tests (not to throw a wrench in our plans, but just thinking about how to best use time with interviewees so we can make decisions based on it).

[chloeatplay]

Thanks @emgollie and also wonder as mentioned earlier today re: how do responses connect with potential future badges? I think we should first make an effort to connect these responses to "attribute" badges in order to have a functional quiz.

[ghami]

I wondered if drilling into the data in criteria in badges as well as potentially trying to encourage use of one of the six Holland Codes, where appropriate, could help link future badges to responses from the quiz. Will be able to add Holland Code info to some of the badges that have come out of the interviews tomorrow. Happy to talk though now as Emily suggests, if you want?

[ghami]

Should have mentioned you both in last comment @chloeatplay @emgollie ! Still getting used to GitHub!

[ghami]

Thanks @chloeatplay and @emgollie for very, very helpful chats over last couple of days! I've made a first attempt at adding Holland Code tags to each of the badges in the pathways doc https://docs.google.com/document/d/1qrHwseb1voltzhr_vDIB80L3jJxHHfUKnMXmZxytyDo/edit#heading=h.iqke3r16eztg to see if I could use them to make useful categorisations. I found I was able to make fairly easy choices but for some I felt needed more than one category. I've colour coded the Holland Codes I've added in terms of using one category, multiple or if I was unsure (colour code key is located above first table of badges). I referenced the Holland Code tagging slide in the initial quiz doc (slide 32) https://docs.google.com/presentation/d/1PHUJzSrB7dQ0rAdHlgzl4Dy2sqqnrCveRs9ZtQkzWn0/edit#slide=id.g2b049a664_06 when I wasn't quite sure and just to check I was using correct categorisation. Now and again I also felt I should reference a more in depth list associated with each category and so referenced http://en.wikipedia.org/wiki/Holland_Codes. I think it could be useful to tweak the Holland Codes. For example, as I was working through the badges, I felt 'influencer' might have been more appropriate than the original Holland Code 'persuader'. I would find it really helpful to know if others find the Holland Code categories associated with each badge as useful / appropriate descriptors and as discussed very happy to try other categorisations or create a new set - will test out some of the others reffered to in https://openbadges.etherpad.mozilla.org/discovery-attributes? tomorrow.

From our discussions, I have also worked on the sections slide (slide 2) on the quiz doc to try to narrow these down and also make them a little more narrative driven (I actually realised these are closer to sections I had set out when I first started thinking through the quiz dev in the pathways doc!). The narrative could be extended through using perhaps some relevant persona / character based imagery for each categorisation as people work through the quiz? Not sure if this is along the lines of what you were thinking for using more narrative @chloeatplay ? I haven't made any more changes to the quiz doc at the moment as I thought it might be useful to keep further changes / devs to the new etherpad for the quiz.

[ghami]

Screen prototype content at: https://openbadges.etherpad.mozilla.org/discovery-quiz

Logic and reduced content (based on feedback from DML user testing) for prototype at https://docs.google.com/presentation/d/16_7SNBsUCUXFHAA0m_Wsz9jTjgu3HAUOpWnL_kU37FY/edit#slide=id.g1d4cccd23_1267

Logic/content doc does include O*Net job families which might be dropped for MVP as they contain industries we're not engaging with yet. If dropped, may need other way to define job industry of interest (eg Tech, Healthcare), to demonstrate why relevant badges from those industries would be presented on screen 3.

[chloeatplay]

Prototype created :tada: https://docs.google.com/a/varelidi.com/forms/d/1T0FqgLKYw0cLUPvxirTZ2UIvOJHVZYtC71cU8jU1I24/viewform Includes 3 sections to test different approaches a) Holland Codes (with animated gifs) b) O*Net & Cross-career badges (sans images) c) Badges (link directly to content). This last section for now is populated with fake data from the pathway doc for the purposes of the quiz.