Plan testing of hypotheses around contribution

[sarawilcox]

What

Decide on approach to testing hypotheses around:

• links to contribution in context
• share buttons

We now have partials encouraging people to contribute at the bottom of pages in:

• the design system
• style guide
• accessibility guidance
• service standard

• @amyj2110, as discussed.

[amyj2110]

Hypotheses

• If we changed the contribution prompt to be a question begging an answer, people would feel invited and encouraged to enter the contribution process. We know this works if we see an increase in people starting a contribution from this prompt
  • For usability test: The question will invite and encourage people to click the link and start contributing. We will see people clicking on the link and telling us they feel like they can and want to click the link.
• If we set expectations of GitHub being part of the contribution journey, a user would feel encouraged to continue the contribution. We know this works, if we hear people tell us it helped them make a GitHub account or we see new commenters on existing issues
  • For usability test: Letting people know about GitHub will set their expectations about what’s going to happen when they click the link (check whether mentioning GitHub puts people off clicking the link, or makes people more confident?).
• If we replace the hyperlink to contribution with an action link so it stands out more, people will see the call to action more easily. We know this works if we see an increase in people starting a contribution from this link
  • For usability test: The display / design of the link will be more noticeable, people will be able to find the contribution section of the page quicker and know what to click.

Others for the usability test

• Understanding
  • How well do people understand the contribution message? (Ask them to read and explain it back to us in their own words)
• Compelled to act
  • Linked to encouraging, how compelled do people feel to click the link and contribute based on what we’ve put?

Are we focussing on this message on certain pages, i.e. service standard, accessibility guidance? (Will need to form tasks for why people would be on these pages)

[sarawilcox]

Thanks @amyj2110. A few more thoughts for the usability test.

• Do people think to go to the bottom of the page and do they recognise the contribution prompts? Do they see them as separate from the page content? (You've covered some of this above.)
• When people click the link, do they recognise that they are on GitHub now and do they see it as a place they could comment?
• We've focused on asking for "research findings". Do people think they have research findings to share? Would they have other things they'd want to share?

We have 4 different messages for different contexts:

• service standard
• design system
• content style guide
• accessibility guidance

I wonder whether we could find participants to test the different messages. Some of them will use more than 1 part of the service manual. The service standard may be more difficult to test but the NHSX service assessment team may be able to help us find some participants for that. (I have contacts there.)

[Indu-Kaila]

• [x] Draft the Discussion guide re the planned testing
• [x] Circulate the Discussion guide to the rest of the team for comment

[Indu-Kaila]

Here's the draft discussion guide for comment https://hscic365-my.sharepoint.com/:w:/r/personal/inka2_hscic_gov_uk/Documents/Service%20manual%20UR%20Draft%20docs%20for%20comment%20(Indu)/Draft%20Discussion%20Guide%20-%20Contribution_ServiceManual_260421.docx?d=w1373fda1b7eb49389b24f4cf5ebbc896&csf=1&web=1&e=rrhzvb

[Indu-Kaila]

One thing I've thought about is I understand that the action link has been newly designed/added to the contribution areas on relevant Service manual pages and that there was a 'normal' link prior to this design. Thinking that click test analysis (Optimal workshop) could have been used rea comparison between the old 'normal' and the newly designed/implemented action link -> This might have already been done (getting my head around ur already carried out in the team)

[Indu-Kaila]

13 ur sessions were carried out on five days between the 12th May and the 20th May 2021. A draft discussion guide was circulated to the team for comment. The final discussion guide is attached here for reference. Final Discussion Guide - Contribution_ServiceManual_260421.pdf