Preference test for Homepage hero section

[kristine-eudey]

Overview

We need to preference test the hero section of the homepage in order to learn whether the current version or newly designed version appeals most to potential and actual users of our site. We then can use these learnings to make a data-informed decision about which to move forward with when the new homepage is launched.

Additional Detail

We have done some user research but the same size was too small. This issue will provide additional data.

Action Items

• [x] Review existing research findings and document in this issue a summary of the findings and the issues of concern, such as small sample size
• [x] Read this article in order to get a refresher/primer/good overview of how best to utilize preference testing.
• [x] Sign up for a free account on Usability Hub.
• [x] Set up a preference test that asks participants to compare the two versions below.
  • (right click image and select "Save Image As")
  • 2 Versions of each are included below: 1 shows the hero in the context of the whole page, and another that shows only the hero section. Use whichever the team determines will be better for testing).
• [x] Present findings to Product and together decide on which version should be developed and launched with the redesigned homepage. Also note findings on this from donor interviews.

1a. Current Hero Image - Full Page


1b. Current Hero Image - Hero Section Only


2a. New Hero Image - Full Page


2b. New Hero Image - Hero Section Only


Resources

Usability Hub: An introduction to preference testing Usability Hub: Sign Up Figma - Homepage Redesign Existing donor research

[github-actions[bot]]

Hi @kristine-eudey.

Good job adding the required labels for this issue. The merge team will review the issue and add a "Ready for Milestone" label once it is ready for prioritization.

Additional Resources:

• Wiki: How to read and interpret labels

[ExperimentsInHonesty]

This issue needs a link to whatever data came out of the donor and user research that is related to this homepage image.

[summerharveyux]

weekly availability: 1 hour Friday night, 1 hour Monday night, 1 hour Tuesday night

[summerharveyux]

12/13 update: just read and typed notes on the article linked above, and have a few questions prepared for the UXR meeting on 12/15. next step: reviewing our existing research findings and documenting in this issue a summary of them. obstacle: only that I need to find those existing research findings! will ask Simone about that now.

[summerharveyux]

provide update

1. Progress: have read the article, will decide tonight w/ the team what images to present
2. Blockers: didn't know where to access existing findings but Simone shared the link with me
3. Availability: Thurs night 5-8pm
4. ETA: after 12/25, since we're taking a holiday break

[summerharveyux]

Update:

1. Progress: I just reviewed existing research findings and document in this issue (see below) a summary of the findings and the issues of concern, such as small sample size.
2. Blockers: I was surprised to find minimum relevant info in the google drive. I was expecting to see something about previous testing of the hero image, such as concept testing maybe, and something about sample size (since that's specifically mentioned in the issue Step hackforla/product-management#1, which made me think something relevant to sample size could be found in the google drive). But, I found minimum info in the google drive relevant to the hero image, and nothing about sample size. That's ok! I can determine what sample size we need in order to obtain statistically significant results, using G-Power. 3: Availability: Mon night 7-9pm, Tues night 5-7pm
3. ETA: expect to have preference testing finished after 12/25, since we're taking a holiday break the week after 12/25 I believe

Summary of existing research findings and issues of concern as found in the Google drive:

Everything relevant extracted from Google Drive Folder ‘UX Research:’ Given Issue prompt of “Review existing research findings and document in this issue a summary of the findings and the issues of concern, such as small sample size:”

• It is relevant that what is most important for volunteers to see on page are (in order): 1) What is HfLA (12/13 ranked this first); 2) Project descriptions; 3) Positive impact of projects; 4) Project social causes; 5) How to join; 6) Open roles; 7) Communities of practice; 8) News; 9) Newsletter
• Users liked the old header image: it gave them a sense of community
• Users preferred to not have the mission in the header
• HfLA mission/description needs to be on top of page: volunteers ranked ‘What is HfLA’ as hackforla/product-management#1 to see on page, and all 9 other nonprofit sites in C&C analysis featured mission statement on top of page
• Users preferred photos over icons to illustrate the program areas: they might also prefer photos over icons (or images that seem similar to icons) for the hero image
• It is important to: legitimize the organization; draw people in; improve brand image
• Volunteers like when there is plenty of white space on page, says that it gives them “room to breathe.” Potentially relevant if 1 of the 2 hero images has more or less white space than the other image.
• Several volunteers have mentioned homepage shouldn’t be too long, and that they wouldn’t scroll very far: keep both hero images the same size.
• Previous (2, I believe) users said ”The hero image doesn’t really give me an impression of what HFLA is about,” and “It’s just that the picture doesn’t really do anything for me.”
• Previous user said “-”Looking at the hero image, I would expect to read the word volunteer. Like maybe instead of saying ‘you can help’ saying ‘you can volunteer’.”
• Previous user liked the (hero) photo, thought it gave a sense of community, and thought the homepage matched their expectations
• Previous user wanted to know whether the photo was of actual people from Hack for LA, and liked the picture, but thought it could be more interactive (she’s a fan of websites with animations)

[github-actions[bot]]

@summerharveyux

Please add update using the below template (even if you have a pull request). Afterwards, remove the 'To Update !' label and add the 'Status: Updated' label.

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either: 1) place your issue in the developer meeting discussion column and ask for help at your next meeting, 2) put a "Status: Help Wanted" label on your issue and pull request, or 3) put up a request for assistance on the #hfla-site channel.

_{You are receiving this comment because your last comment was before Monday, December 27, 2021 at 11:19 PM PST.}

[summerharveyux]

1. Progress: discussed the preference test language with the team tonight. updated the preference test to reflect that new language, and shared the link with the team for distribution. distributed it to several UX-relevant slack groups (1) HfLA general; 2) Research Ops, 3) Hexagon UX, 4) UXPA Boston, 5) UX Research and Strategy, 6) People Nerds, 7) UX Rescue, 8 ) Ideate Labs, 9) Delta CX). will be keeping an eye on responses and manually stopping data collection once we reach N=115. will then review the results upon completion and prep to share them with the team.
2. Blockers: None
3. Availability: 2 hours Wed night, 3 hours Thurs night
4. would love if we reach 115 by our next team meeting on 1/12! if not, I'll distribute to slack groups again for more responses
5. N/A

[summerharveyux]

Progress: Discussed in 1/12 team meeting that we have N=104 participants so far (that I collected since last week's meeting by posting multiple times in different UX slack groups), and it's split 51%/49% for the two images (lol). Next steps: getting for ~11 more participants so we achieve our goal sample size of 115 (even though I don't think it'll change the results much). compiling the qualitative data for the 2 images, identifying themes, sharing those results at next week's team meeting. Blockers: N/A Availability: 2 hours Wed night, 2 hours Thurs night ETA: Next week's team meeting on 1/19 Pictures:

[summerharveyux]

Progress: Shared in 1/19 team meeting final results (https://docs.google.com/presentation/d/1imm90Dve3ngSAAa2jEVfwGFMQkXtNvKThLkf8iRYl2I/edit?usp=sharing), since we reached N=116 participants and I was able to create affinity diagrams for all the qualitative findings. Next steps: Simone asked me to add my notes to the presentation so she can share it with Bonnie on Sunday (since I cannot attend that meeting), and asked me to add my miro board to the team miro board Blockers: N/A Availability: 2 hours Wed night, 2 hours Thurs night ETA: EOD on Thurs 1/20

[ExperimentsInHonesty]

Notes for @summerharveyux

• I love that the deck uses the image on each slide to communicate.
• It would be good if each slide (that can) had an image of what you are trying to illustrate (placement of header text is the example that comes to mind)
• This deck needs to use a template specific to this project (see this issue https://github.com/hackforla/knowledgebase-content/issues/29)
• The insights are clear and very easy to understand with minor exceptions, such as, I was not clear how many people preferred faces and how many people preferred not to see faces (which is different from preferring the hands image because it's more polished)
• I could be wrong, but after hearing and seeing these findings, it sounds like we need a new image altogether.

Thanks for making the presentation for Simone to show us.

[summerharveyux]

Progress: talked w/ Simone at weekly meeting about Sunday's presentation, and what my next steps should be. I understand my next steps are to update the slide deck to address Bonnie's bullet points, so it's a reference for designers: make it clear what users liked and didn't, so they can pick a new hero image.

Next steps: 1) reach out to Anya via slack to get the ball rolling on that approved issue (not relevant to this issue, just typing this for myself). 2) update the slide deck, and upon completion share it w/ Simone and save in the google drive

Blockers: N/A

Availability: 2 hours Wed night, 2 hours Thurs night

ETA: EOD on 1/26, or by weekly 5pm EST meeting on 2/2

[summerharveyux]

@sacamp I have a question about one of Bonnie's bullet points above. Her 3rd bullet says this desk need to use a template specific to this project, but the issue she linked doesn't provide a powerpoint template (that I'm seeing). Is there a template I should be using? I can model my deck after the other deck in the 'Research Presentations' folder in the Homepage Redesign Research folder, if that is what is needed. Thank you!

[summerharveyux]

Progress: Addressed all of Bonnie's comments on the slides. Spoke with Isaac about presenting to design team on 2/3, was later reminded by Shruti that there's no meetings the first week of the month, rescheduled w/ Isaac to present to design team on 2/10. link to edited slides: https://docs.google.com/presentation/d/1lFY4bGDYicZDIbj5CNxhiiNGNq479cOOj3-pbD_ifqM/edit?usp=sharing

next steps: present to design team on 2/10.

blockers: none

availability: 2 hours Wed night, 3 hours Thurs night

[summerharveyux]

Progress: ready to present to design team on 2/10. link to edited slides: https://docs.google.com/presentation/d/1lFY4bGDYicZDIbj5CNxhiiNGNq479cOOj3-pbD_ifqM/edit?usp=sharing

next steps: present to design team on 2/10.

blockers: none

availability: 2 hours Wed night, 3 hours Thurs night

[summerharveyux]

Progress: presented to the design team on 2/10. Bonnie was at this meeting, and we decided to prep materials for and subsequently run another preference test (see new issue: https://github.com/hackforla/website/issues/2763). this new preference test will be between two new images (if I understood correctly): an image Bonnie has from a pre-COVID, in-person meeting, and an image that will be created of stitched together screenshots of zoom meetings.

we are unsure if 1) people will feel naturally more attracted to an in-person meeting and less attracted to the idea of yet another zoom meetings, contributing to increased solo and isolating work, or if 2) people will like that the zoom screenshots indicate that anyone can join HfLA from anywhere, rather than needing to be based in LA, and that the zoom screenshots convey our flexibility and resiliency in response to covid. To answer this question with data instead of assumptions, we'll conduct another preference test.

Next steps: just assigned myself to the new issue. will comment only on that new issue from here on out, since this preference test issue is completed.

Blockers: see new issue Availability: see new issue