Touchpoints design and hallway study

[AlexandracmONRR]

Next Steps:

• [x] Email NPS about how they are tracking pages (Lindsay)

• [x] Have meeting with NPS

• [x] Email FWS about how they decided to choose the questions they did (Alex)

• [x] Message Haley about DOI guidance on feedback mechanism (Lindsay)

• [x] Email GSA developer, Ryan, about page tracking (Alex)

• [x] Discuss what we have found with the team

• [ ] Plan a hallway study on touchpoints design

[AlexandracmONRR]

Response from Ryan at GSA Touchpoints:

• Thumbs up/down is only available in the template. But the question should be editable. If not, let's chat!
• Tracking responses across each page: embedded forms should track the host and path from where the feedback was submitted (across pages)
• The button and modal are scanned for 508 accessibility and compliance and use USWDS accessible components. We do have a couple improvements and enhancements coming to improve accessibility, specifically around the modal. You can tab to the tab button and trigger the modal via keyboard, and we've had our Section 508 team review the feedback flow as well.
• Touchpoints doesn't offer much in the way of customization, but happy to chat. Typically, agencies can use CSS style overrides on their time.

Response from Haley (DOI): DOI does not have any guidance/guidelines. ODDD team should let them know what we are doing and our process.

[AlexandracmONRR]

Response from FWS:

1. Your team’s process for developing your website’s feedback survey, including some questions about interaction with FWS employees, trust in FWS, etc. We use OMB's a-11 Section 208.7 (page 4), specifically the customer experience drivers as the basis of our questions. We recently adopted OMB's updated a-11 survey thumbs up/thumbs down template, and dropped the question about interaction with employees because it's not necessarily relevant on a web form. We added optional questions about which user group the member is a part of, and what they were looking for when they arrived. The text box provides a free form place for them to describe their feedback.
2. Is your survey being tracked by page or by the site as a whole? We track this on every national wildlife refuge and national fish hatchery page. We also use this form on national visit us pages as that indicates interest in trip planning. We have a separate survey running on all other pages, but it hasn't been updated to the latest thumbs up/down survey style yet.
3. Your installation of the button for feedback and if you did any customizations. We're importing the US design standards CSS file, which is a survey form dependency. We haven't customized the front-end beyond that. Check out the delivery page (attached screenshot) of your survey for how to include the form on your site.

[AlexandracmONRR]

Discussion with NPS:

NPS is a HISP, so they have been doing a survey for a long time. Previously they contracted with a company. The contractor would collected numbers, do analytics, and compare to other agencies. Eventually, they ended the contractor agreement and went over to Touchpoints.

Details of using Touchpoints:

• Touchpoints has easy data collection
• Touchpoints allows for limited questions (that have already been approved). For new questions, OMB must approve, which could take up to 18 months. It is very cumbersome process and requires the digital team to work with the Information Collection person in the agency, form questions, then fill out a form with all the details of these questions.
• Touchpoints doesn't not have an analytics component, so they had to build their own in PowerBI and share in SharePoint.

Touchpoints forms that NPS uses:

• They have found one of the most useful template forms is "Was this page helpful?" with a yes/no answer that leads to an open text for more explanation.
• They also have a form that is for CX; it is an 18 question form that is triggered after a user navigates through 3 pages.
• Gave us link to SharePoint site with analytics and email to person responsible for installing "Was this helpful" on nps.gov

Other advice:

• Get user research volunteers through volunteer.gov

[AlexandracmONRR]

General synopsis:

Accessible: yes, has been tested and uses USWDS components. Tracks pages: yes, embedded forms should track the host and path from where the feedback was submitted (across pages). Questions: OMB has some already approved questions that do not require approval or PRA approval.

• OMB has a larger CX survey - NPS triggers this after a user has clicked through 3 pages
• OMB has a short/simple "was this page helpful?" question that has a yes/no answer and then an open ended answer.

Style: style can be customized using the agency's CSS style. Data collection: easy data collection that occurs in the touchpoints app. Data export and analysis: this requires more work and to set up a system of collection and then analysis. NPS used PowerBI and SharePoint. PowerBI was needed since they have a lot of responses and they have parks that need filtering to their specific slices of nps.gov, so may not be as applicable to ONRR's processes.

[AlexandracmONRR]

Questions in OMB CX survey:

1. I am satisfied with the service I received from [Program/Service name].)
2. This interaction increased my confidence in [Program/Service name]. OR I trust [Agency/Program/Service name] to fulfill our country's commitment to [relevant population].
3. Anything you want to tell us about your scores above? (free text)
4. My need was addressed.
5. It was easy to complete what I needed to do.
6. It took a reasonable amount of time to do what I needed to do.
7. I was treated fairly.
8. Employees I interacted with were helpful.
9. Anything else you'd like to share with us? (free text)

[cthomasONRR]

@AlexandracmONRR @lpgoldstein Thanks for sharing! I appreciate getting to see all the OMB question options. Of them these are the only ones I think would be relevant to us:

5. It was easy to complete what I needed to do.
6. It took a reasonable amount of time to do what I needed to do.
7. Anything else you'd like to share with us? (free text)

I'm definitely ok with just the "Is the page useful: Y/N" and free text though!

[AlexandracmONRR]

Next sync up we can discuss our main goal of implementing touchpoints on onrr.gov and hallway testing. For now, this issue is complete.

[AlexandracmONRR]

Our original questions that we discussed:

• Your overall satisfaction of this site: thumbs up/ thumbs down
• Did you find what you were looking for? yes/no
• Please explain what you were looking for: (open text)
• Do you have any suggestions to improve the site?
• Do you want to participate in user research?
• [Maybe a question about who they are as a user]

Other question options we discussed:

• Did you complete your task?
• What task were you trying to complete today?