search

codefordurham / school-navigator

Navigate the Durham, NC public school system
https://durhamschoolnavigator.org
26 stars 16 forks source link

User testing of the school profile page #336

Closed basiacoulter closed 7 years ago

basiacoulter commented 8 years ago

It would be nice to decide on the design of the school profile page by testing alternative designs with users.

basiacoulter commented 8 years ago

Suggested steps for A/B testing:

I. Identify parent networks to approach (both Bendte and I may have access to parent networks of the Triangle; I can, for example, get connected to a FB for SoDu Parents Posse.

II. Create an opt-in form (maybe using Typeform?) for parents to opt in to participate in A/B testing

III. If the number of opt-in is sufficient, divide the participants into 2 groups:

IV. Send a link to the school profile page prototype to test participants: design 1 to group A, design 2 to group B.

V. Ask participants to fill out a survey:

  1. Based on this page, can you quickly formulate a decision whether or not you’d consider this school for your child?
  2. Can you quickly find information you need to consider if this school is a good choice for your child?
  3. If you answered not to question 2, what information are you having trouble finding?
  4. Can you easily tell what categories of information are offered on this page?
  5. If the order of information on this page helpful?
  6. If you answered ‘no’, to question 5, how would you change the order of information?
  7. Is any information you’d consider important missing from this page?
basiacoulter commented 8 years ago

Alternatively ask more specific questions to see if users really are able to find this information on the page such as:

  1. When does the school day start and end for this school?
  2. How many miles is this school from your house?
  3. Does this school have a lottery process?

In this scenario, since we will not be observing the users directly, the concern is they might be able to accomplish each task but with difficultly. So they may respond correctly but struggle to find information on the page, and we will not know it. Maybe we could combine the more specific questions with rating for each task like so:

  1. When does the school day start and end for this school?
  2. How hard was finding this information on the page? (scale 1-5)

And at the end, we could still get the users' input on suggested adjustments which might allow us to get to most frustrating aspects of the design.

basiacoulter commented 8 years ago

@bendte I have been thinking more and more about the user testing for the school profile page and here are my reflections:

basiacoulter commented 8 years ago

Next steps:

sarahsba commented 8 years ago

@basiacoulter, here is the list of contacts that I've started: https://docs.google.com/spreadsheets/d/1ixon2Rw_-KeFBlxzTWmuNY-2UOrkxUBOOli1J5IrUC4/edit#gid=1875312278. Let me know if you have any questions. These are all people who are already aware of SN and have been in touch with us before, so we'd probably want to use a different outreach email for them than for new folks. I can work with @npage915 on these.

Excited for this!

basiacoulter commented 8 years ago

Thank you @sarahsba! I just sent a request for permission to edit. I will create a new tab in the same spreadsheet where we can start adding contacts for people who have not participated in past surveys.

@npage915 Looks like we need two different emails: one will less explanation of the project for people who have participated in the past; and one with more explanation. Both have to have an enticing sentence or paragraph that will make them want to opt in :-)

@bendte @LASHirsh As soon as Sarah gives me editing permissions, I will invite you both and you can start adding contacts that you come up with to the tab for new participants.

sarahsba commented 8 years ago

Great. You should have editing permission now. Let me know if it didn't work!

On Aug 13, 2016 2:02 PM, "basiacoulter" notifications@github.com wrote:

Thank you @sarahsba https://github.com/sarahsba! I just sent a request for permission to edit. I will create a new tab in the same spreadsheet where we can start adding contacts for people who have not participated in past surveys.

@npage915 https://github.com/npage915 Looks like we need two different emails: one will less explanation of the project for people who have participated in the past; and one with more explanation. Both have to have an enticing sentence or paragraph that will make them want to opt in :-)

@bendte https://github.com/bendte @LASHirsh https://github.com/LASHirsh As soon as Sarah gives me editing permissions, I will invite you both and you can start adding contacts that you come up with to the tab for new participants.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/codefordurham/school-navigator/issues/336#issuecomment-239633659, or mute the thread https://github.com/notifications/unsubscribe-auth/ALfAvrhVYnV6FZBhWXCVIuR6P9YoWBovks5qfgbQgaJpZM4I5OKV .

basiacoulter commented 8 years ago

@bendte @LASHirsh @npage915 @sarahsba, I got connected to a woman who's an admin for the SoDu Parents Posse on FB. She loves what we do and she'll post a link to the opt-in form on their FB page. So this would be new participants opt-in. What needs to happen for us to decide if this form is a go, so I can pass it on to her?

basiacoulter commented 8 years ago

@alexlemann What needs to happen to get a "go" for posting the opt-in form above on a curated parental page on FB?

basiacoulter commented 8 years ago

@npage915 Can I share this opt-in form with a FB parental group? https://codeofdurham.typeform.com/to/Si0GHy

jtf621 commented 8 years ago

Basic,

I am OK with sharing the Opt In form as you suggest. Thanks.

Jeremy

On Sep 7, 2016, at 8:54 AM, basiacoulter notifications@github.com wrote:

@npage915 https://github.com/npage915 Can I share this opt-in form with a FB parental group? https://codeofdurham.typeform.com/to/Si0GHy https://codeofdurham.typeform.com/to/Si0GHy — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/codefordurham/school-navigator/issues/336#issuecomment-245271112, or mute the thread https://github.com/notifications/unsubscribe-auth/ACNKTenScdzPGOFfIVtq2MYg_4IEiRcgks5qnrPrgaJpZM4I5OKV.

npage915 commented 8 years ago

Thanks for putting the form together, Basia! Looks great.

On Wed, Sep 7, 2016 at 8:57 AM, jtf621 notifications@github.com wrote:

Basic,

I am OK with sharing the Opt In form as you suggest. Thanks.

Jeremy

On Sep 7, 2016, at 8:54 AM, basiacoulter notifications@github.com wrote:

@npage915 https://github.com/npage915 Can I share this opt-in form with a FB parental group? https://codeofdurham.typeform.com/to/Si0GHy < https://codeofdurham.typeform.com/to/Si0GHy> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ codefordurham/school-navigator/issues/336#issuecomment-245271112, or mute the thread https://github.com/notifications/unsubscribe- auth/ACNKTenScdzPGOFfIVtq2MYg_4IEiRcgks5qnrPrgaJpZM4I5OKV.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/codefordurham/school-navigator/issues/336#issuecomment-245271933, or mute the thread https://github.com/notifications/unsubscribe-auth/ASw8D9WRcFsqRmn3nBNpCBLZCDhn_Ayzks5qnrStgaJpZM4I5OKV .

basiacoulter commented 8 years ago

Great.

Nicole can I draft a paragraph explaining the project or do we already have something like that?

Basia

Sent from my iPhone

On Sep 8, 2016, at 2:06 PM, npage915 notifications@github.com wrote:

Thanks for putting the form together, Basia! Looks great.

On Wed, Sep 7, 2016 at 8:57 AM, jtf621 notifications@github.com wrote:

Basic,

I am OK with sharing the Opt In form as you suggest. Thanks.

Jeremy

On Sep 7, 2016, at 8:54 AM, basiacoulter notifications@github.com wrote:

@npage915 https://github.com/npage915 Can I share this opt-in form with a FB parental group? https://codeofdurham.typeform.com/to/Si0GHy < https://codeofdurham.typeform.com/to/Si0GHy> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ codefordurham/school-navigator/issues/336#issuecomment-245271112, or mute the thread https://github.com/notifications/unsubscribe- auth/ACNKTenScdzPGOFfIVtq2MYg_4IEiRcgks5qnrPrgaJpZM4I5OKV.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/codefordurham/school-navigator/issues/336#issuecomment-245271933, or mute the thread https://github.com/notifications/unsubscribe-auth/ASw8D9WRcFsqRmn3nBNpCBLZCDhn_Ayzks5qnrStgaJpZM4I5OKV .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

npage915 commented 8 years ago

Hi Basia -

This is the description from the Code for Durham website, which I think will suffice:

Durham School Navigator https://durhamschoolnavigator.org/#/ is web application tool to help parents understand the different public schools in Durham County and make informed schooling decisions. Durham School Navigator provides a common and easy-to-use platform for all public schools to share information about themselves with parents. The mapping web app provides a way to

In addition the school profile survey to school principals, provides a common and easy-to-use platform for all public schools to share information about themselves with parents. Explore the sample school profile here https://durhamschoolnavigator.org/#/school/247/

We have partnered with Durham Public Schools (DPS) to have them directly link to durhamschoolnavigator.org vs. the current mapping tool.

Durham Charter Collaborative, Durham PTA Council, and DPS are supporting Code for Durham in launching the School Navigator and the principal survey for the school profiles.

Let me know if you think this will work!

Best,

Nicole

On Thu, Sep 8, 2016 at 2:17 PM, basiacoulter notifications@github.com wrote:

Great.

Nicole can I draft a paragraph explaining the project or do we already have something like that?

Basia

Sent from my iPhone

On Sep 8, 2016, at 2:06 PM, npage915 notifications@github.com wrote:

Thanks for putting the form together, Basia! Looks great.

On Wed, Sep 7, 2016 at 8:57 AM, jtf621 notifications@github.com wrote:

Basic,

I am OK with sharing the Opt In form as you suggest. Thanks.

Jeremy

On Sep 7, 2016, at 8:54 AM, basiacoulter notifications@github.com wrote:

@npage915 https://github.com/npage915 Can I share this opt-in form with a FB parental group? https://codeofdurham.typeform.com/to/Si0GHy < https://codeofdurham.typeform.com/to/Si0GHy> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ codefordurham/school-navigator/issues/336#issuecomment-245271112, or mute the thread https://github.com/notifications/unsubscribe- auth/ACNKTenScdzPGOFfIVtq2MYg_4IEiRcgks5qnrPrgaJpZM4I5OKV.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/codefordurham/school-navigator/issues/336# issuecomment-245271933, or mute the thread https://github.com/notifications/unsubscribe-auth/ ASw8D9WRcFsqRmn3nBNpCBLZCDhn_Ayzks5qnrStgaJpZM4I5OKV

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/codefordurham/school-navigator/issues/336#issuecomment-245689793, or mute the thread https://github.com/notifications/unsubscribe-auth/ASw8D9a55acgcQZmmiwhInro7goZRGnEks5qoFE-gaJpZM4I5OKV .

basiacoulter commented 8 years ago

It is bit too long for a FB post, but it's a great starting point. I will shorten before passing on.

basiacoulter commented 8 years ago

Here is what I am passing on to post:

Join Code for Durham in making Durham School Navigator (https://durhamschoolnavigator.org) a better tool for parents to make informed schooling decisions! Opt in to participate in usability testing of the web application they are building for you! https://codefordurham.typeform.com/to/Si0GHy

Durham School Navigator is an easy-to-use web application:

basiacoulter commented 8 years ago

Btw, anyone on FB with parent friends in Durham, please post that as well in your FB feed!

basiacoulter commented 8 years ago

@bendte @LASHirsh Will you be able to some to the meetup on Sept 20? Would be great if we could finalize the usability testing plan. Define user tasks; design the survey; design an in-person session. We already have 1 volunteer for in-person usability testing!

bendte commented 8 years ago

@basiacoulter Yes, I can meetin on Sept 20. I've been working on a survey draft and in-person questions too. That is awesome that there is 1 in-person usability testing volunteer!

LASHirsh commented 8 years ago

@basiacoulter I do plan to attend the 9/20 meetup and agree that we should finalize our usability testing plan! I, too, have been working on a survey draft and on an in-person moderated testing script. See you Tuesday!

basiacoulter commented 8 years ago

Fantastic. Thank you @bendte and @LASHirsh, and see you next Tuesday!

bendte commented 7 years ago

@basiacoulter Caktus was locked when I got there this evening, so I figured it might have been canceled for tonight. I noticed your post about Victor finishing up some work. Is this page (https://durhamschoolnavigator.org/#/school-new/183/) totally final?

I based by first survey draft on your mock-ups and several items are missing from this page, so I wanted to be sure that this page is absolutely final (as the map isn't working for me) because some of the questions will need to be changed.

Here is a first draft for the survey and usability test although, please note, some questions will need to change if the page is totally final. Also, the draft is in Gdocs, but I like Typeform and also read about some cool survey implementations via Survey Monkey (which I believe would need some code embedded in the page). @basiacoulter @LASHirsh

https://docs.google.com/document/d/1OrfikRlKBcEDTo8kwi67cQTJU-QHijbnNnHh1iuC9ko/edit#

LASHirsh commented 7 years ago

Hi @basiacoulter. Just saw your note. Yes, in retrospect, it makes sense that the meeting would have been cancelled for City Camp. :-)

@bendte, nice work on the survey/script draft. :-) I see you used a modified/shortened Krug's script for the in-person testing, too. ;-)

For the sake of simplicity, I suggest we work off of your Google doc to complete the draft. What I had put down was actually remarkably similar, although I do have an additional/alternate task we might consider and a couple of comments/suggested edits. Would you be willing to give commenting access to the document to the team?

I know we discussed using Typeform for the survey, and think we all agreed we like the tool. Based on what you read, though, do you see any particular advantages that you think warrant us switching to Survey Monkey?

basiacoulter commented 7 years ago

@bendte, I was there until 6:54pm, and @victor-rocha left just a little earlier, so I am not sure how we missed you. The lights were on and we were in the front classroom. But yes, I did not leave the doors open, because there were only the two of us and I did not want anyone to go unnoticed. I guess I assumed anyone would see the lights, peak inside and knock on the window. Sorry about that.

The entire content of the profile page has been transferred into the prototype page, and organized into tabs. I can see it all. What is it you're not seeing?

Thanks for working on the drafts! I'll take a look in the next few days.

bendte commented 7 years ago

@basiacoulter I was basing the survey/usability test questions off of the mock up that you did a few weeks ago and not the actual prototype page. I think it's more that a few items have moved around from the mock up to the prototype (map is now down at the bottom left and isn't loading for me). I'd love to talk through the changes from the mock up to the prototype before the survey/usability testing. I think I also don't know much about the history of this project (school survey and which fields are optional, database structure) which may help me too. It is looking great!

@LASHirsh Thanks for letting me know about the permissions. I just changed them, so the comments should be open. I really like Typeform and they probably have a tool similar to Survey Monkey. I just took a survey on Durham Public Library's website and it was great because the survey was embedded on the page, so you could keep browsing the site while also taking the survey (instead of having to switch back and forth between the site and the survey). Just wanted to toss out that idea.

For next steps:

  1. Feedback on the survey/usability questions.
  2. Should we plan on meeting on October 4th to finalize things or meet before then?

Thanks!

basiacoulter commented 7 years ago

Ok, @bendte, it does seem like we need to talk. I had done 3, I think, iterations of mockups (including the hand sketches), and they all had been done before our card sorting exercise. The purpose of the card sorting exercise was to determine the best way of organizing the page's content in a tabulated scenario. I did not make a mockup for that scenario, I jumped directly into coding it. So, I apologize if I did not communicate more clearly that I was going to base the prototype on how we organized the content during card sorting.

We did not explicitly say where the map should go in the card sorting exercise. I included it with the location piece. I have gone back and forth were exactly to put it. There were issues with any position. I am hoping that usability testing will give us some insight into where it should go. The fact it is not loading is not related to the page itself. I've had that problem when viewing at home as well. There are many issue with the map.

The only section that I placed differently than what we came up with in card sorting was parent involvement. We had placed it under "Who we are," which is what I did first, and felt like it was out of place. If you feel strongly it should be under "Who we are" we can move it back there and let users tell us what makes sense to them.

Now the fact that specific information may be missing for any given school is not the matter of the page itself but what information which school provided. We have to identify a school that has as much information filled out as possible. So we can work on that next time and for testing we'd use a link to a school that does have the most complete profile. Like this one: https://durhamschoolnavigator.org/#/school-new/247/

I am attaching the photo of card sorting as a reference here as well. 10939728-5591-11e6-8ef2-e56c415be147

basiacoulter commented 7 years ago

Oh, one more thing. In the card sorting above, we placed location/zoning under At a Glance (that tab is called Overview; I did not get a sense we finalized the wording for the tabs, so this is still an open topic I think, for example we changed Testimonials to Community Feedback). That's what I interpreted as the map. Since the address etc are grouped together with school hours in the original profile page and I did not feel comfortable breaking up information from within each section, all that information ended up in the hero part of the page where we listed contact info and school hours during card sorting. According to the card sorting outcome, the map (if interpreted as location) should be in the top left of the Overview tab. But when I put it there, it interfered with page scrolling. I just happened to always have my cursor above the map when trying to scroll down, and instead of scrolling, I kept zooming into the map. So I think the map itself has some usability issues that may be worth addressing but beyond that I feel like I placed the content as described in the card sorting diagram above. So we definitely should go over it all in person.

basiacoulter commented 7 years ago

And I am really sorry @bendte for any confusion I may have created. Another data point showing why communicating person increases chances of shred understanding :-)

bendte commented 7 years ago

@basiacoulter I think it's all due to Mercury Retrograde (which ends tomorrow, hallelujah)! :) Looking forward to getting together soon to talk everything over in person. The tabs look so great. Excited about everything coming together.

basiacoulter commented 7 years ago

@bendte I finally had a chance to look at the document. You did a TERRIFIC job! I think I see the points where information is not on the page. I think it was my fault that in mockups, and maybe even in the card sorting, I did not faithfully use categories of information that the profile page actually currently captures, but the ones we listed during the civic hacking day based on interview responses. So I think there are two categories of info that simply are not captured on the profile page, and we will have to adjust that. I have made comment. Beyond that it's fantastic. See you on Tuesday?

npage915 commented 7 years ago

Hey team -

I'm so sorry but I can't come again tonight. This term has been pretty unreal, however, it's coming to an end (woo!) so hopefully I'll be able to start showing up regularly again soon. That said, I was thinking that, if helpful, I could do some user testing with some of the families who are in school with me (we have a few parents with school-age children) - it could be a good perspective as many of them will have recently/are currently going through the process of choosing a school.

Let me know if you'd like me to do so, and if so, I may set up a call with Bendte or Basia to review the process of how I should do the testing and what info I should collect. Also, let me know if there's anything else I can support remotely.

Thank you so much! Nicole

On Sun, Oct 2, 2016 at 5:41 PM, basiacoulter notifications@github.com wrote:

@bendte https://github.com/bendte I finally had a chance to look at the document. You did a TERRIFIC job! I think I see the points where information is not on the page. I think it was my fault that in mockups, and maybe even in the card sorting, I did not faithfully use categories of information that the profile page actually currently captures, but the ones we listed during the civic hacking day based on interview responses. So I think there are two categories of info that simply are not captured on the profile page, and we will have to adjust that. I have made comment. Beyond that it's fantastic. See you on Tuesday?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/codefordurham/school-navigator/issues/336#issuecomment-250998066, or mute the thread https://github.com/notifications/unsubscribe-auth/ASw8D97yEg02N0X6tc6_tSE-KyHuGPysks5qwCUjgaJpZM4I5OKV .

bendte commented 7 years ago

Yes, I will be there tonight. See you then!

basiacoulter commented 7 years ago

@npage915 it would be great if you could run some usability testing. I think @bendte, @LASHirsh, and I have to clean up the script and scenarios based on the actual page content, and then it'll be ready to go. When would you be available foe a call with us?

npage915 commented 7 years ago

Hi Basia -

I have a lot of flexibility early next week. Can you let me know a few times that could work for you?

Thanks! Nicole

On Tue, Oct 4, 2016 at 3:02 PM, basiacoulter notifications@github.com wrote:

@npage915 https://github.com/npage915 it would be great if you could run some usability testing. I think @bendte https://github.com/bendte, @LASHirsh https://github.com/LASHirsh, and I have to clean up the script and scenarios based on the actual page content, and then it'll be ready to go. When would you be available foe a call with us?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/codefordurham/school-navigator/issues/336#issuecomment-251481551, or mute the thread https://github.com/notifications/unsubscribe-auth/ASw8D7MdCOMlUbMy3C3UA5hEZ02o7ZmCks5qwqLLgaJpZM4I5OKV .

basiacoulter commented 7 years ago

@npage915 Do you think you'll be able to come to the next Meetup? We have finalized the survey and usability test scenarios but there is still work to be done before we can actually do the testing, so if you were planning to be at the meetup on 10/18, we could talk then. Next week is a bit busy for me. But if you're not able to be there on the 18th, I might be able to talk to you on Wed evening. Let me know.

basiacoulter commented 7 years ago

Done. Recommendations are being implemented.