Sitewide Public Websites and Facilities research ticket templates are updated

[xiongjaneg]

User Story or Problem Statement

As a researcher on Sitewide Public Websites and Facilities teams, I need research ticket templates that meet current researchers' needs and processes.

Acceptance Criteria

• [x] Existing research ticket templates have been reviewed.
• [x] Research ticket templates have been updated to meet current researcher needs and processes:
  • [x] Research plan
  • [x] Conversation guide
  • [x] Prep tasks
  • [x] Research sessions
  • [x] Synthesis and reporting
  • [x] Readout and post tasks
  • [x] Discovery
  • [x] Collab cycle
  • [x] Runbook
• [x] Feedback from others has been incorporated, and changes have been merged:
  • [x] Research plan
  • [x] Conversation guide
  • [x] Prep tasks
  • [x] Research sessions
  • [x] Synthesis and reporting
  • [x] Readout and post tasks
  • [x] Discovery
  • [x] Collab cycle
  • [x] Runbook
• [x] Ensure the following item on the template(s) is still marked Optional: Research readouts to weekly Monday design meeting
• [x] Ensure there is still (or if there isn't currently add one)a step in the template(s) that "It should be reviewed by the Sitewide UX Lead." (wording here not verbatim)

Team

Please check the team(s) that will do this work.

• [ ] CMS Team
• [x] Public Websites
• [x] Facilities
• [ ] Accelerated Publishing

[cindymerrill]

5 points for me? Depends on how many times we meet together and what's involved in updating the templates with all of my picky tasks. Could potentially expand to 8 points.

[ALogsdon3]

I think 5 is good for me, especially if @cindymerrill wants to get the ball rolling. I want to lowball it a little since #14515 may get unblocked during this sprint.

[xiongjaneg]

Thank you both! I'll put 5 on for now. If any info changes, please let me know and we can adjust. And if it moves into next sprint, we can adjust as well.

[mmiddaugh]

@ALogsdon3 I was reminded earlier about the opportunity to leverage previous research findings available in the repo and/or those which might be available in the design system. I don't know if this is universally relevant enough to add to the template but I wanted to mention it. I'll leave it to you all to determine how to incorporate it, if at all.

[ALogsdon3]

Thanks, @mmiddaugh. I'll tag @cindymerrill in here because she's taking the first big swing at this ticket. I'm inclined to think of reviewing past research and existing guidance in the design system as its own type research, which would be covered under discovery, but we haven't yet finalized a plan for how granular we plan to make these tickets. It will be good to keep that on our radar! Also cc: @davidmpickett since they're also involved in this process.

[davidmpickett]

Currently "Reviewing previous research" just gets a passing mention in the overarching Research Runbook ticket template.

It could be helpful to draft a ticket template for Research Discovery / Pre-work, aka any steps in the process that routinely happen before creating a Research Plan. That would, by necessity, be more loosely defined than the other ticket templates, but might help for awareness and planning purpose to call it out as a discrete part of the process. Could include things like:

• Align with Product Manager & Product Owner on the high-level purpose and scope of the research
• Review Research Repo for previous relevant research
• Explore tooling options (thinking about all the work Cindy did for PACT Act research)
• Other ideas?

[cindymerrill]

I like that idea, @davidmpickett ! Want to draft something to put into our ticket template branch?

Could you say something more specific than "tooling"? Do you mean something more than prototyping?

[davidmpickett]

First draft of a discovery ticket - https://github.com/department-of-veterans-affairs/va.gov-cms/blob/1621df8711d7a1dcb3b23c3e56294d26fddf5324/.github/ISSUE_TEMPLATE/research-discovery.md

[cindymerrill]

That looks great, @davidmpickett ! Is that in the same branch that I'll be working on revisions in? I haven't had time to start yet...

[davidmpickett]

@cindymerrill Yup! All in the working branch

[ALogsdon3]

@davidmpickett the discovery ticket looks great!

[cindymerrill]

Yesterday I updated the Conversation Guide ticket template and shared it with @davidmpickett and @ALogsdon3 to review: https://github.com/department-of-veterans-affairs/va.gov-cms/blob/research-runbook-updates/.github/ISSUE_TEMPLATE/research-conversation-guide.md

Dave thought it looked great and didn't have any feedback. Still waiting for feedback from @ALogsdon3 , and then would like to merge in (with help from @davidmpickett ) so we can use the template today to create a ticket for the Header/Footer project.

FYI @FranECross @jilladams

[ALogsdon3]

@cindymerrill this looks great! I have one suggested change:

FROM "Use the appropriate template for the type of research you are conducting:" TO "Use the appropriate template for the group you're conducting research with"

Rationale: Less ambiguously signaling that the distinction between the two templates is audience, not methodology.

[cindymerrill]

Thank you for your feedback, @ALogsdon3 ! I made the revision, so now I'm all set to merge with @davidmpickett . YAY!

[FranECross]

@cindymerrill Just a head's up that I added the following two items to this ticket's AC just to be sure. Thanks!

• Ensure the following item on the template(s) is still marked Optional: Research readouts to weekly Monday design meeting
• Ensure there is still (or if there isn't currently add one)a step in the template(s) that "It should be reviewed by the Sitewide UX Lead." (wording here not verbatim)

cc @jilladams

[cindymerrill]

@FranECross @jilladams I'm responding to your two additions above...

• Ensure the following item on the template(s) is still marked Optional: Research readouts to weekly Monday design meeting

I can add that this is optional when I get to revising the relevant template; I don't know how it's currently stated.

• Ensure there is still (or if there isn't currently add one) a step in the template(s) that "It should be reviewed by the Sitewide UX Lead." (wording here not verbatim)

I made this more general ("UX Lead") for the conversation guide ticket I just updated in case these ticket templates are used by others outside Sitewide, but I could change it to "Sitewide UX Lead" if that's preferred.

FYI @davidmpickett @ALogsdon3

[cindymerrill]

I worked with @davidmpickett to merge our ticket template changes so far--the most important of which was the conversation guide ticket template. It doesn't include the change that @FranECross requested above, so that will have to wait for the next version.

[cindymerrill]

@FranECross @jilladams I just updated the conversation guide ticket template to add "Sitewide" in front of "UX Lead", and also made a few more small updates. And got more practice in merging a pull request, too, which is not an intuitive process for me (yet).

[cindymerrill]

Hi @davidmpickett @ALogsdon3 ! I just made an updated version of the research plan ticket template: https://github.dev/department-of-veterans-affairs/va.gov-cms/tree/research-runbook-update3. It's very similar to the conversation guide ticket I already finished. Could you please review it and share your feedback here?

[cindymerrill]

Added the list of specific research ticket templates to the Acceptance Criteria above.

[cindymerrill]

@davidmpickett @ALogsdon3 I also revised the research prep tasks ticket template: https://github.dev/department-of-veterans-affairs/va.gov-cms/tree/research-runbook-update3. Could you please review it and share your feedback here?

Note that I didn't do much revising of the Editor-facing research section, so @davidmpickett should review that section carefully and probably share with Jo for input.

[cindymerrill]

@ALogsdon3 @davidmpickett Here are more direct links to the two templates ready for review:

1. https://github.com/department-of-veterans-affairs/va.gov-cms/blob/research-runbook-update3/.github/ISSUE_TEMPLATE/research-plan.md
2. https://github.com/department-of-veterans-affairs/va.gov-cms/blob/research-runbook-update3/.github/ISSUE_TEMPLATE/research-prep.md

[ALogsdon3]

@cindymerrill Looks great! Here's a few things that stood out to me in the Research plan template: Suggested workflow/tasks section: can we indicate that these two steps are specific to Usability testing? Identify moderator(s), notetaker(s), and observer(s) Identify time slots for research sessions (twice as many slots as # of participants desired)

[ALogsdon3]

@cindymerrill On the research prep template: I wonder if we should separate out the steps that are relevant to moderated and unmoderated studies. Right now, there's a subset of tasks within the Veteran-facing research section that would be relevant to unmoderated studies, but quite a few more that are not relevant. These are the ones that don't seem relevant to unmoderated studies:

• [ ] Review Perigean's recruitment survey before kickoff meeting

• [ ] Attend kickoff meeting with Perigean and Sitewide UX Lead and/or PM

• [ ] Test Zoom login from Perigean

• [ ] Change display name of Perigean Zoom account to "<Moderator's first name> (Moderator)"

• All of Notetaking prep

• Most of Pilots

• Most if not all of Recruitment/scheduling

It seems like the cleanest way to present this in a template would be to duplicate the veteran-facing research section and label one For veteran-facing moderated research and the other For veteran-facing unmoderated research, and delete the tasks that don't apply from the second one. Then we could add the link that's under Field Research and Unmoderated Studies (https://depo-platform-documentation.scrollhelp.site/research-design/planning-unmoderated-studies) to the For veteran-facing unmoderated research section.

If we agree this is a good way to proceed, then I'd just suggest one more edit: In the ACs, I would note that second one is specifically for moderated studies, like so: "For moderated studies, the moderator(s), notetaker(s), and observer(s) have been invited to research sessions and daily debriefs"

[cindymerrill]

@ALogsdon3 There are lots of kinds of research we can do, and usability testing is only one of them, so I wouldn't call out usability testing specifically. With unmoderated research, it is true that some of the detailed steps I listed aren't applicable (e.g., all the calendar-related ones), but many are (e.g., you might want a kickoff meeting with Perigean, you could review their recruitment email, and you should run pilots). Rather than creating separate sections for unmoderated vs. moderated research, which would make a much longer ticket template, I'd recommend editing any new tickets you create to delete anything that doesn't apply (and add anything special that you need). I could add notes about what is only for moderated research (or unmoderated), but that would probably add a lot of words to the template. Are there specific tasks that you'd like added in order to better cover unmoderated research?

[cindymerrill]

@ALogsdon3 I just updated the research-plan template to add "if moderated research" to the 2 tasks you listed: https://github.com/department-of-veterans-affairs/va.gov-cms/blob/research-runbook-update3/.github/ISSUE_TEMPLATE/research-plan.md.

I'll see how doing that might look in the prep-tasks ticket...

[cindymerrill]

@ALogsdon3 In the prep-tasks ticket, I added "(for moderated research only)" after the section title "Recruitment / scheduling sessions". I believe that most of the other tasks apply to both moderated and unmoderated research, with a few exceptions (e.g., the two zoom-related tasks). If I were to make a separate unmoderated research section, I'm not sure that I want to copy all the specific steps detailed in the Platform documentation page because they might get outdated, but I could I guess if that would be helpful. Since you're planning unmoderated research, maybe we could talk about how to best support that with this ticket template. I'm also curious about what @davidmpickett thinks. https://github.com/department-of-veterans-affairs/va.gov-cms/blob/research-runbook-update3/.github/ISSUE_TEMPLATE/research-prep.md

[ALogsdon3]

@cindymerrill I think these are good compromises. I agree that it's not ideal to let the template get super long, and your most recent edits mitigate the issues I was concerned about effectively.

[cindymerrill]

@davidmpickett doesn't have time to give feedback on the research-plan and research-prep-tasks ticket templates this week, so (with his consent) I went ahead and merged my changes so @ALogsdon3 and I can use the new ticket templates right away for upcoming research planning/prep work. If/when Dave has time later, he can share his feedback and more revisions can be made. FYI @jilladams @FranECross

[cindymerrill]

Revised research-plan and convo guide ticket templates in a minor way, and then did major revamps of 3 more ticket templates:

1. Research sessions
2. Synthesis and reporting
3. Readout and post tasks

I plan to review them tomorrow and then share with Alexis for feedback.

[cindymerrill]

Revised ticket templates and then shared with @ALogsdon3 and @davidmpickett for review:

1. research-plan (minor revisions re Sitewide UX Leads gets feedback from PO and provides approval)
2. research-conversation-guide (minor revisions re Sitewide UX Leads gets feedback from PO and provides approval)
3. research-sessions — major revision
4. research-synthesis-and-reporting — major revision
5. research-readout-and-post-tasks — major revision

[ALogsdon3]

@cindymerrill cc: @davidmpickett These look fantastic! Most of my comments are on the Readout template and none are very major.

Research Sessions template

Reviewed research-sessions: thumbs up, no notes

Research Synthesis and Reporting

Reviewed research-synthesis-and-reporting and have one small suggested edit. In the Collaboration with Sitewide UX Lead ACs, I think it makes more sense to order the feedback tasks like this:

• [ ] Research report feedback requested
• [ ] Sitewide UX Lead requested feedback from VA PO and shared with researcher
• [ ] Research report feedback incorporated

And I wonder whether the first one should be more specific--who are we asking for feedback here? the UX lead? Anyone else?

Research readout and post tasks

Reviewed research-readout-and-post-tasks:

User Story "AS A researcher I WANT to wrap up my research study SO THAT I can share it insights with others and then move on protect participants' private information" Rationale: demonstrates value beyond just sharing (it's the insights!) and reflects the full range of activities in this template more accurately.

Research Readout(s) section

• Should we add/update to include UX Lead to this one?: "Work with your UX team, Product Manager, & Product Owner to generate, refine, and categorize recommendations"

• Add "(optional)" to this item: "(optional) Consider doing a practice runthrough to gauge flow and length" Rationale: keeps this parallel with other "consider" items in the list

Destroy and Archive Research Materials

• Suggest edits to this item: "Once your readouts are complete, you should destroy delete any session recordings and transcripts from your computer saved to your local machine" Rationale: more concrete and specific. Maybe add something about emptying your trash folder, too?

ACs

If you agree with the last change, I'd also edit the last AC: "Session recordings and transcripts deleted from your local machine computer"

[cindymerrill]

Thank you so much for these detailed edits, @ALogsdon3 ! I agree with all of them except maybe one (discussed below). Some of them pertain to wording that was in Dave's original runbook tickets that I didn't rewrite, and I think your updates really help :).

The only edit I'm not sure about is your reordering of these AC's:

• [ ] Research report feedback requested
• [ ] Sitewide UX Lead requested feedback from VA PO and shared with researcher
• [ ] Research report feedback incorporated

I was thinking that maybe Amanda would provide some feedback herself, and then we'd update the docs before she shares them with the PO, but I don't know if that's the case. I don't want the template to be too prescriptive because it may work one way sometimes and the other way other times. Maybe it's more logical to order it the way you have suggested, regardless of what ends up happening. Thoughts?

[cindymerrill]

On the other hand, I already have an AC for "All feedback on research report incorporated", so I might just delete the previous one that seems duplicative.

[cindymerrill]

All updated now @ALogsdon3 !

I'd like to merge the changes before the end of the sprint, so I'd like to get feedback from @davidmpickett before that, if possible.

[cindymerrill]

Incorporated @ALogsdon3 's feedback on 3 tickets; @davidmpickett will hopefully have time to review them next week.

Today I also revised the last 3 research ticket templates and shared with Alexis and Dave for feedback. (All feedback could be shared here in the ticket.) Additional notes from slack copied below...

• Do you think we still need the review cycle research ticket template? It’s about going through the Collaboration Cycle, so I renamed it to research-collab-cycle. Seems like the PM or Designer usually drives this, but sometimes it’s the Researcher who brings up that we need to do it. I’m open to removing it and putting the Collab Cycle info elsewhere, but if you think it should be separate, here it is with my revisions so far: research-collab-cycle
• I just did only light revisions to the research-discovery ticket
• Also made light revisions to runbook--ux-research. If we decide to delete the research-collab-cycle template, then that line should get removed from this template. There are still “ADD LINK AS LAST STEP” notes, which I assume I should do after merging all of my revisions?

[cindymerrill]

@ALogsdon3 @davidmpickett I would like to merge changes by the end of the sprint on Tuesday, if possible, so ideally I'd have your feedback by then. If not, let me know when you'll have time to review them, and if it's going to be a while, I might merge anyway.

[ALogsdon3]

• "Do you think we still need the review cycle research ticket template?" I think incorporating it into the appropriate templates makes sense rather than having a separate template for it.
• "I just did only light revisions to the research-discovery ticket" I trust your judgment; I didn't review what changes you made, but I recently used this template and it seemed sufficiently broad to cover most discovery scenarios.
• "There are still “ADD LINK AS LAST STEP” notes, which I assume I should do after merging all of my revisions?" That makes sense to me! cc @cindymerrill @davidmpickett

[cindymerrill]

Thank you for your input, @ALogsdon3 !

[cindymerrill]

@ALogsdon3 @davidmpickett Which ticket do you think should get the collab cycle review info? Maybe prep-tasks, or perhaps earlier while the research plan is being developed?

[davidmpickett]

I just committed a few changes, mostly specific to Editor-facing research.

I do think it's worth keeping the Collab Cycle ticket template in here. I just reworded it to make it a little clearer what the specific purpose is and the division of labor between PM and Research Lead. Because researchers are often at the vanguard of a project, they are likely to be in a situation to have to prompt PMs to create initial Collab Cycle tickets. This isn't perfectly cleanly separated from the information in the Research Prep ticket, but I think this is enough of an improvement for now.

[cindymerrill]

@davidmpickett Have you reviewed ALL of the ticket templates? So if I agree with all of your changes, I could merge to the main branch?

[davidmpickett]

Yes, I looked at all of them. You and Alexis already had them in great shape, so wasn't really much for me to do. :)

[cindymerrill]

@davidmpickett Thank you for making your revisions right in the branch, and they all look good to me. I merged them, and then added the ticket links into the runbook template. So now the research ticket templates are all done and ready to use! FYI @ALogsdon3 @FranECross @jilladams