Idean Design Review

[petergault]

• How do we explain the tools during the onboarding?
• How do we integrate the apps into a coherent flow? Here is the problem: We've built a series of tools for teachers to use to learn writing skills. However, unlike say the tools you would use to build a table (saw, hammer, sander), our tools are not used to build a single end product. Instead, they are somewhat isolated - you can do some sawing, some hammering, some sanding, but they are not all being used towards one single goal.
• How do we articulate to teachers how they use these tools in one coherent flow?
• A compounded issue here is that some of the apps work differently than others. For example, grammar hero features random content.

  Here's what you can do with Quill:

  1. Play Grammar Hero and improve grammar knowledge.
  2. Play Quill Proofreader and asses your grammatical knowledge.
  3. Play WordLike and expand your vocabulary.
  4. Play Quill Writer by writing a story together with a friend using a shared list of words.

[petergault]

Big Questions for Idean

Quill was previously a series of two grammar tools. We are now making the jump from grammar tools to a platform for writing tools. We will do so by launching two new tools focused on storytelling and vocabulary. Over time we will introduce tools focused on sentence construction, essay writing, and research, among others.

Our large concerns are:

1. How do we communicate that we are a platform for tools?

We will need to communicate to teachers that we have a range of tools available. They should know how each tool works.

• Where will we educate the teachers?
• We can put together a series of slides and then link to it from the homepage, show the slides during sign up, and perhaps reference it in the app.
• Will this be sufficient for introducing the tool?

2. How do we ensure teachers know how to implement it?

Teachers will need to use our activity planner to assign activities.

• How should we go about thinking about how we can test this interface?
• How can we guarantee that teachers understand how to implement our tools?

3. What’s a simple way of enabling teachers to use these tools?

Right now we let teachers select individual levels or select a package of levels. Is this the best way of getting teachers on board?

• We could, for example, let teachers simply select how long they want to spend on each tool.
• One big problem here is that each tools levels are somewhat dissimilar from the other.

4. What’s the best way of ensuring that students understand the tools?

We can show students an explanation before they start the game, or we can provide a help menu that brings up a tooltip. Let’s make sure that students understand it.

5. How do we ensure that teachers understand the data produced?

Let’s take a look at the data reports produced.

Other issues:

1. A Gameplan for Gathering Feedback

I'm interested in thinking about how we can do better testing sessions and get better feedback. At this point we have built up a network of 5,000 teachers, and next year I'd like to work with that community to improve our product. For example, at one point we went to a school that recently stopped using our product, and that feedback was 10x better than the feedback we receive from the people who love our product. I'd like to put a gameplan together for how we can effectively test and receive feedback.

2. Testing WordLike

We're working on a few new apps, and one of them is called WordLike. You can play it now at wordlike.org. This is a great example of a prototype that's ready for meaningful feedback. We could put together a game plan for how we can go about soliciting feedback in a meaningful way. I think there is a very fun element buried in this game, but there still needs to be some serious iteration on it.

3. Onboarding as a Platform

As I said before, we currently have 2 apps, and over the summer we will launch 3-5 more apps. This introduces a serious design problem as we need to teach each of our teachers about how each of these apps work. When we have only have one choice available, it's easy to explain the experience. As soon as introduce choice, we also take on the burden of explaining those choices.

To solve this problem we're going to tackle it from two angles. The first is to have a strong onboarding process, where we track every single step of the onboarding process, and display it as a list to the teacher. This will help us and them keep on track, and we can provide hints to get them through these steps. A key part of this onboarding process will be walking them through a brief overview of the apps.

The second way in which we are going to solve this problem is creating pre-created lists of activities for teachers. This will give teachers a package of content this is ready to go.

[liyajames]

Guys I'm putting some thoughts here but let's discuss it at our call today because every answer I put I have some question about your specific context... This is all words, so we might try some stuff out sketching in the sessions with Alice etc.

1. How do we communicate that we are a platform for tools?

Slides aka intro sequence is good for introducing the experience. But that's not where people learn how to use tools. They will get a broad stroke understanding of "oh these things has 5 main features and this is why they're important for me." To augment the learning experience on the tools, you might also link to examples and stories of how people use the tools from the the tool's landing page or home.

2. How do we ensure teachers know how to implement it?

I've actually designed something like this for Common Core curriculum before. We ended up mapping our content against the common core in a visual way. Since teachers usually go out and look for content when they get to certain teaching modules, we map it in a way that there's very clear. You already have this in a list format on your site today. But we find that the younger the teachers teach, the more visual they are. I think it'll also be good to let them bookmark activities so they are doing lesson planning. Looks like you're turning up on page 1 when I search for some of the common core modules so that's great!

3. What’s a simple way of enabling teachers to use these tools? To answer this, we have to think about the teacher's workflow and mental model:

• Using the tools as homework
• Using the tools as activities in classroom
• Is there another context?

Both of these instances, students should be able to self-pace, so I think they should give out tools and not time or levels. We can discuss more. I suggest this because from my past research, the thing teachers struggle the most with is that students in the classroom is actually not all on the same level, esp in charters like Montesouri etc.

4. What’s the best way of ensuring that students understand the tools? YES! Even we needed instructions when we played the games, they were pretty anxiety provoking at times. In our experience, you need def to show an explanation, then the game, then a variable types "YAY! You did it!" screens. We need to tap on intrinsic and extrinsic motivations.

Good to refer to Hooked Model for intrinsic/extrinsic motivation. We've used this to model out gamification for a variety of apps that gets people to keep upleveling and engaged.

5. How do we ensure that teachers understand the data produced? I looked at the current reports which are in table format, in my experience, teachers are very visual and they are also triaging all the time. So I highly encourage us to help them triage 2 things: which students need more support. What trends can we see in mistakes (a lot of them are looking for changing their techqniues if it's not working, in some ways homework serves this purpose as well).

Other issues:

1. A Gameplan for Gathering Feedback

This I can't describe on here but Alice has done a lot of research and can give you a framework to plan it. Alice - in general I suggest always incorporating some feed-forward research and and feed-back in every session. I would def start with qualitative studies first, then maybe a mix (qual+quant) using usertest.com.

2. Testing WordLike

We played it and agree it can do a bit more handlholding. I tested it on a few people here and they're a bit confused even as adult.

Brain fart: I wonder if we can't add some characters to these and make them even more game like. clicking on the character, he gives you tips.

3. Onboarding as a Platform

This seems like a great approach. I think it will help but at some point, you might consider rethinking the IA of your site so it flows a bit better to support the concept that this is a platform...

[alicegrantham]

Hello all,

Some follow up from our discussion yesterday

1. Intro for usability study We've Google Doc'd a test script for task-based usability study - http://is.gd/TestScript.
Can be adapted for validation studies using any prototype. The intro language is great for setting expectations and reassuring the participant that we're testing the work and not them. We can provide an example discussion guide that can be adapted for discovery research with teachers later this week.

2. Heuristic evaluation There's a lot about this topic online but at a high level, we can evaluate design according to these 10 heuristics http://www.usabilityfirst.com/usability-methods/heuristic-evaluation/

3. Insight mapping I've transcribed my post-its onto a Google Drawing http://is.gd/insightMap. The center part, moving down is loosely based on a possible teacher flow over time. In the four corners of the drawing, are peripheral nuggets from our conversation.

We're looking forward to putting a clear gameplan together with you on Friday and keeping the dialogue open and ongoing that you're getting what you need from our engagement.

Let me know any adjustments you'd like for our Friday agenda:

• Map our notes/insights with current pages
• Looking for highest impact initiatives, and prioritizing
• Create the game plan for tackling teacher on-boarding (or other high priority)
  • i.e., signup, getting started, using, and getting value from data
  • "Game plan" should include
  • what we want to learn (research goals)
  • how we'll learn it (research method, tips, practice?)
  • how to organize and use findings (synthesis)
  • plan for iterating on design (prototype testing, trello)
  • considerations for timing and other constraints