Website Carousel Text A/B Test

[majkush]

Overview

The carousels for the website prototype need to be updated due to the transition away from data-literacy modules that is taking place in the project. An A/B test of the carousels will be performed to optimize their usability.

Action Items

• [x] Gather the overarching goal for the carousels and the total number of carousels needed
• [x] Define a goal for each carousel
• [x] Craft a research plan for the A/B test with a UX Researcher
• [x] Create a Google Doc to house copy
• [x] Draft version 1 copy for each carousel
• [x] Draft version 2 copy for each carousel
• [x] Get feedback from team, stakeholders, and the organization
• [x] Give to designer to add copy to Figma for web and mobile prototypes
• [x] Get approval for research plan
• [x] Set up the study
• [x] Recruit participants
• [x] Execute study
• [x] Evaluate results
• [x] Report results and suggestions for which carousels were preferred
• [x] Work with a designer to have the simple carousels updated into the latest version of the Figma.

Resources/Instructions

• UX Writing Resources
• Current website prototype for desktop

  Completed Work

• website carousel text draft
• Carousel A/B Test Research Background and RQs
• Presentation slides on findings
• Open-ended responses about how we can improve accessibility to open-access data

Key Takeaways

• Through this research and a team conversation of results at the 11/30/2022 Demo Day ATD meeting, we determined that the Simple carousels will be used in the product. They were updated into the hi-fi wireframes (mobile and desktop) on 12/14/2022 and the issue was closed.

[majkush]

Note: The action items are being developed. What is on the issue at the moment is a draft. I will meet with Lucy to discuss further in the next few days.

[majkush]

@steffiekim and I are making progress. We met and discussed the first draft of the text. Steffie suggested adjusting the text so that it would work better with the A/B test design method. As a result, I created two sets of text, one that reflects the Empowerment brand principle and one that reflects the Simple brand principle. Text was then put into drafts on Figma to create visuals. Steffie is working on setting up the research study.

[steffiekim]

Drafted a survey experiment on Qualtrics and shared it with @majkush (the whole experiment items here, preview here) Participants will be randomly assigned to seeing either 4 empowering messages or 4 simple messages If resources allow, additional questions from a previous survey that wasn't launched can be added.

[mxajPrice]

A/B test comments from PMs (@lrchang2 , @bkong , @mxajPrice):

• Would be useful to define "open-access data" at the beginning of the survey
• Can a "back" button be added?
• Where it asks "I am willing to click on "Learn more"", switch phrase to "Learn more button" because in the screenshot it is a little hard to read "Learn more."

Question: How many participants are you looking to fill this out?

[steffiekim]

A/B test comments from PMs (@lrchang2 , @bkong , @mxajPrice):

• Would be useful to define "open-access data" at the beginning of the survey
• Can a "back" button be added?
• Where it asks "I am willing to click on "Learn more"", switch phrase to "Learn more button" because in the screenshot it is a little hard to read "Learn more."

Question: How many participants are you looking to fill this out?

Hi @mxajPrice , here are the answers:

1. I added the definition at the beginning of additional questions after the experiment. Here's @majkush's definition: Open-access data refers to data that is freely available for anyone to access, use, download, analyze, copy, or redistribute.
2. Yes. The back button is added now.
3. Relevant items are modified.
4. We're aiming for at least 40 participants. But as always, the more, the better.

[lrchang2]

Potential Recruitment Strategies

LinkedIn Post and shared by team members (LinkedIn post seems to have been deleted but it was used for #68)

Reddit subreddits:

• [x] r/SampleSize
• [x] r/SurveyExchange
• [x] r/takemysurvey

UX communities:

• [x] Hexagon UX Slack channel #user-research-requests
• [x] mixedmethods Slack channel #recruitments
• [x] ResearchOps Slack channel #take-part-in-research
• [x] User Research Academy Slack channel #take_part_in_research

Hack for LA community:

• [x] #general channel
• [x] #access-the-data channel

Social media:

• [x] LinkedIn - Matt to post to LinkedIn, tag HackforLA, and then ask rest of team, via Slack, to share.
• [x] Twitter - Matt to post to Twitter, tag HackforLA, and then ask rest of team, via Slack, to share.

[steffiekim]

Finalized recruitment language and confirmed with @lrchang2: :wave: Hi! We are UX researchers from Hack for LA working on developing LA City’s website for open-access data. :globe_with_meridians: Please help us make open-access data easily accessible to all by taking a short 3-minute survey! :bangbang: Here is the survey. Thanks! P.S. If you know someone else who may be interested, please share this survey with them. :slightly_smiling_face:

[steffiekim]

Posted recruitment messages on 3 subreddits and 4 UX communities on Slack. The checked items in this comment show the specifics.

[steffiekim]

To get more participants, I reposted recruitment messages on 2 subreddits (r/SampleSize, r/SurveyExchange) and 3 Slack servers (Hexagon UX, mixedmethods, ResearchOps). r/takemysurvey doesn't allow reposts within one month and User Research Academy Slack had my last post as the most recent one, so didn't want to spam their channel.

[steffiekim]

• Finished data analysis.
• Made presentation slides for reporting the findings.
• Sorted out themes from open-ended responses about how we can improve accessibility to open-access data

[bkong]

@steffiekim @majkush Can you update this issue with links to the A/B test results and the presentation you gave on demo day? Also, please outline next steps or blockers you're waiting to resolve here.

[majkush]

@bkong Absolutely.

@steffiekim and I added a section titled 'completed work' in the issue where we listed our completed work, including the presentations and the Excel spreadsheet of qual data. I will reach out to @deepamohan22 early next week to seek her assistance in getting the approved carousels imported into the latest Figma prototype files.

After that is complete, the issue will be complete and I will close the issue.

[majkush]

Closing the issue. Great work everyone!

@deepamohan22 moved the 'simple' carousels into the Hi-Fi wireframes for both mobile and desktop. They are now ready for @lrchang2 to move them into the current wire-frame page.