sstrassberg
commented
1 year ago @gforbes4 Overall, looks good. A couple things to consider:
- For your recruitment criteria, do you want ideally 30 completed sessions? If so, then you should request 1.5x that amount. If your ideal completed sessions is less than 30, you should state that.
- For your available time slots, please include 2x the total of requested participants. This allows Perigean to quickly slot for noshows or reschedules.
- For your demographic criteria, please refer to this recruitment submission form for the baseline demographics you can request. Everything else you currently list should be turned into Screener questions that participants can answer to filter for those specific criteria.
gforbes4
Research_Plan_CT_SMF.md
Research Plan for gcio-vfep (SMF feature enhancement)
Background
This research covers the proposed redesign and enhancement of the "Specialized mission filter" feature (SMF) in the VA's GI Bill Comparison Tool (CT) product. A usability study will be organized to gauge the updated feature's ease of use and overall value to the user. Enhancements include:
The CT product is publicly available (accessible whether or not the Veteran is signed in to VA.gov). It is not a new product, though modifications to it may occur from time to time to enhance the user experience.
Link to product brief
OCTO Priorities
This research supports and aligns with the OCTO priority goal of enhancing Veterans’ personalized online experience.
OCTO priorities
Veteran Journey
The filtering feature to be studied is part of the GI Bill Comparison Tool, which provides information and guidance toward educational and training opportunities for Veterans.
For many Veterans, this can be an important step in the "Starting Up," "Reinventing Myself," and "Engaging VA to access benefits and services" segments of the Veteran journey.
Research Goals
The primary goal of this research is to confirm ease of use and gauge the level of understanding of how the Specialized mission filters can be applied and updated in the GI Bill Comparison Tool.
Research questions
Questions we seek to resolve by observing how participants use the SMF feature include:
- How filtering options are understood: Do participants seek additional information on selections by locating and opening the "Learn more" accordion control to expose descriptions of the filters?
- How are the filters selected when using the GI Bill Comparison Tool: Do participants choose a single filter or (as now supported) select multiple filters?
- Are participants aware that they need to choose the Update results button to refresh results?
- Are participants satisfied with the updated results?
- Are participants aware that they can modify the filters and update results at any time?
- Do participants have any questions about other available filters or filter groups?
- Are sight-impaired or other participants with accessibility needs in any way hampered from using and modifying filters?
### Hypothesis This study is intended to provide insight into whether or not CT filters, in particular those included in the SMF, are being used as designed and are providing Veterans with an effective, efficient means of discovering the opportunities they seek. Notes and recordings will be collected and analyzed, and the study will be considered successful if no serious impediments are discovered as participants work through the primary feature being studied, the feature functions as expected, and no design changes are deemed necessary.Methodology
This research will be a moderated usability study focusing on the specialized media filter feature in the GI Bill Comparison Tool web application.
Each session will include note-taking and, with the permission of the participant, recording, to determine how participants use and apply the filtering features under study.
Additional resources:
Location
Study participants will be interviewed remotely one at a time using the Zoom application. "After hours" sessions will likely need to be scheduled for most if not all participants.
Research materials
The [Conversation Guide](url goes here)] will serve as the primary source for interactions with participants during each session of this study. A final script will be ready for review at least two days prior to deployment of the repositioned filters on the GI Bill Comparison Tool staging server.
Recruitment
OCTO works with Perigean, a small business, to handle the recruitment, scheduling, and compensation of participants.
Recruitment approach
The intended audience should comprise Veterans who are currently or may in future be seeking educational or employment training opportunities.
Read this introduction to inclusive research and use the recruitment checker (google sheets) to understand OCTO's targets for inclusivity.
Recruitment criteria
Total number of Veteran participants: 30
Demographic criteria: Veterans, preferably those who are actively seeking or will in future be seeking education or training opportunities and may want to review institutions that support selected specialized mission types listed in the filter.
Current specialized mission institution types are:
Special needs criteria: Recommend 20% of participants be included in the study to ensure design and content is accessible to Veterans with sight, hearing, or other impairments.
Timeline
Research review artifacts, including design materials and this Research Plan, will be submitted to Perigean within two days of the SMF feature's deployment on the staging server.
Preparation
A pilot session with one mock participant will be conducted within two days of the SMF feature's deployment on the staging server.
Availability
Session scheduling: TBD
Team Roles
Other observers and note-takers may also attend but should not be required.