Open quis opened 6 years ago
quis
commented
6 years ago This comment is a placeholder to summarise the work in these pull requests:
The original request to go live page:
| Before | After |
|---|---|
 |
  |
| Before | After |
|---|---|
 |
 |
| Before | After |
|---|---|
 |
 |
quis
commented
5 years ago We did some guerrilla usability testing of this journey and in 1 of 6 (17%) of sessions we saw someone skip straight past the checklist page because of big green button syndrome. While 1 in 6 people would normally be a small number1 in the context of a usability testing session, it’s enough to cause a big workload for our team (assuming it is the sole cause of people not completing the items on the checklist).
The initial reason for using the tick cross pattern for the checklist was:
There’s been some interesting discussion on the GOV.UK Design System backlog2 about users failing to complete items in the task list. A few people have tried different patterns for communicating that items in the task list still need ‘completing’.
So we tried using the task list pattern on the request to go live page. The task list is adapted from the one in the design system in that:
We often check that a service has an appropriate text message sender as a condition of them going live. We don’t mention this anywhere.
The services for whom GOVUK is definitely not an appropriate sender are those in local government. As we have more of these teams starting to use Notify, we should streamline the process by making this check automated.
So we also added a check for this to the task list, for teams who:
| Before | After |
|---|---|
 |
 |
https://github.com/alphagov/notifications-admin/pull/2250
With the caveat that looking only at task completion and quantifying qualitative research can both be problematic and the intention here is to show that the numbers are close enough to say that they could be symptoms of the same problem. Leisa Reichelt’s Mind the Product talk is good on this stuff.
https://github.com/alphagov/govuk-design-system-backlog/issues/72
quis
commented
5 years ago Placeholder comment for the work in this pull request:
| Before | After |
|---|---|
 |
 |
quis
commented
5 years ago Placeholder comment for the work in:
| Before | After |
|---|---|
|
 |
This is the page behind the new item in the task list:
quis
commented
5 years ago Placeholder comment for the work done in:
quis
commented
5 years ago Something about how we’re managing organisations now…
quis
commented
5 years ago Dealing with users who request to go live but haven’t completed all the steps still represents a significant support overhead for our team. We’ve made some improvements to the percentage of incomplete requests with a better page design, but ultimately because it still shows the button people think it’s OK to press the button while some of the items on the page still say [Not completed].
We can do this now because organisations are in the database, which means we can mark the agreement signed as soon as we get it back, without having to deploy code.
—https://github.com/alphagov/notifications-admin/pull/2907
| Before | After |
|---|---|
|
 |
quis
commented
5 years ago At the moment, accepting the data sharing and financial agreement looks like this:
The full process is:
Let's not do that any more.
When the first service for an organisation that doesn't have the agreement in place is in the process of going live, then they should be able to accept the agreement online as part of the go live flow.
Where the checklist shows the agreement as [not completed] then they can follow a link to download it (as happens now). From here, they should then also be able to provide some info to accept it. The info that we need is:
Version - because we version the agreements occasionally, we need to know which version they are accepting. It may not be the latest one if they downloaded it a while ago and it took time to be signed off
Who is accepting the agreement - this will often be someone in the finance team, and not necessarily a team member, so we should let the person either accept as themselves, or on behalf of someone else. If it's on behalf of someone else we need to the name and email address of that person so we have that on record. Obvs if it's them accepting it themselves, we have that already.
We then need to replay the collected info back in a sort of legally binding kind of way pulling in the organisation name too.
Google docs sketch:
User need
As a Notify user I need to easily find information about how to request to go live, So that I’m able to see what I need to go before I can go live and navigate back to it again once I’m ready
Insight
The user journey to request to go live is frustrating for users. In lab testing we saw users get confused when they were directed to the ‘Using Notify’ page. They commented that they didn’t know where they’d been linked to and tried to understand where the ‘Using Notify’ page was in the top nav. There were concerns that they wouldn’t know how to navigate back to the page again if they wanted to.
Once users had performed an action on the go-live checklist, they frequently struggled to navigate back to the request to go live page. In some cases, the only way they could work out how to find the ‘Request to go live’ page was to try sending a message again in order to encounter the error message about trial mode.
In general, users struggled to find the information about trial mode/going live on the ‘Settings’ page. They typically clicked around for quite a while before finding it in ‘Settings’.
The process is not clear to users. In support tickets they say stuff like:
User need
As a Notify user, I need to understand what I must do before I’m allowed to go live, So that I can go live as soon as possible after making the request, without needing additional support from the Notify team.
Insight
In lab testing, understanding of the ‘request to go live’ checklist was mixed. Most understood it was a list of things they need to do before they can request to go live. But one person didn’t pay attention to the checklist at all and just clicked the green button:
‘I saw the big green button saying continue….I guess continue suggests next action….these links up here look optional’
‘The team member hyperlink looks like it might tell me what a team member is generally, not that I have to do something’
Although most understood they needed to add a team member to their service, they typically assigned the wrong level of permission. This was either because they hadn’t read this on the previous page or couldn’t remember the level of permission mentioned.
Support ticket analysis findings May-July 2018
Needing to add or amend the email-reply-to address and/or text sender name were most common things missed off checklist:
Some people use their personal email address as a reply-to by mistake. Others use an inappropriate shared email that we think their users won't recognise, or an invalid address. When it comes to text message sender names, in most cases we're letting them know they can change it from the default to something more relevant for their service.