Strategy for Unmoderated User Testing

[DarcieF]

As a Product team we need the capability to continue user first approaches at all times including unmonitored capabilities through the system itself.

NOTES:

• Explore options for tooling
• A/B Testing, Heat Maps, etc.

[DarcieF]

Goals for OPL:

• Flexible tools that can help collect data that is specific to our needs
• Seek to understand UXD recommendations on tooling
• Create a Spike for testing options
• Develop strategy for unmoderated testing that looks at holistic perspective
• Heat mapping, A/B testing
• Need to define our testing goals then pick the tool
• Is there an Open Source analytics tool - would likely require more dev work

Notes from UXD team:

• Across all product teams at Red Hat we don't have analytics in place
• Many Red Hat products have been established so creating a new data tracking method is hard
• OPL can leverage analytics because it is a new build
• Most of the free or open source tools require developer to set up all tests not as easy for product team to do themselves
• Some tools charge per response of users in addition to the user seat for subscriptions
• Each tool has it's own use case and goal - what questions we have in mind
• User Zoom provides research partners they will assign a researcher to the team / sort of outsources the research
• If we have to choose one tool UXD team recs Loop 11

Tool options:

• Google Analytics - designed as a marketing tool hard to customize for UX needs
• Loop 11 - can send link to study for people to do on their own https://www.loop11.com/
• Clicktale - good for user data that can inform insights https://www.clicktale.com/
• Optimizely - works well with Clicktale to focus on A/B tests https://www.optimizely.com/
• Adobe Analytics - RH has an account but hard to implement https://www.adobe.com/analytics/web-analytics.html
• Qualtricks - powerful survey tool RH has account can make popup surveys, asking questions of people https://www.qualtrics.com/
• Pendo - good for easier quick questions RH teams using https://www.pendo.io/
• Drift - chat bot for directing people to the right topics RH teams using https://www.drift.com/
• Amazon Mechanical Turk - help to recruit users for technical people https://www.mturk.com/
• Optimal Workshop - card sorting https://www.optimalworkshop.com/
• Lookback - screen recording for people to do on their own https://lookback.io/
• User Zoom - unmoderated for all these different types of tests / many of the above but costly W-UX team is getting account with them https://www.userzoom.com/

[lallmon]

@heyjerrybecker This is definitely blocked by #36 knowing the things we would want to measure towards success, would help inform the unmoderated testing we would ues.