taceybadgerbrook
commented
2 years ago Closed taceybadgerbrook closed 2 years ago
taceybadgerbrook
commented
2 years ago taceybadgerbrook
commented
2 years ago The Listen to audio page has attracted 1,161 users since the exhibition opened on 20 October to 27 November, making this by far the most popular exhibition guide for this exhibition probably because of convenience as well as for accessibility reasons.
taceybadgerbrook
commented
2 years ago Use of the QR code Gallery 1 is tracked in GA by campaign IPSExhGuide. 82% of users who accessed the audio guides page did so via the QR codes.
taceybadgerbrook
commented
2 years ago Looking at the guides accessed, the graph below shows a drop off after the introduction which is in keeping with analysis of previous QR code usage.
Guides which appear to have been unusually unpopular having been used 6 or less times are numbers 17, 19, 20 and 29. This might be because the titles were not sufficiently enticing: Eyes on overtime, What We Wore archive, Let Me Be Seen and Acknowledgements.
This analysis of audio guide usage is possible because the QR codes have been set up so that activation of different exhibition guide elements appears in the url. Those who accessed the audio guides without using the QR code show up as having accessed the homepage only because there is no separate tracking associated with the audio guides. If tracking was set up for each audio guide played regardless of whether a user was a QR code users or not, we might be able to see whether QR code users are more engaged, ie listening to more audio guides than the non-QR code using group.
taceybadgerbrook
commented
2 years ago Audio player interactions by QR code users by stop Id are shown in the charts below.
These charts don't actually tell us much more than we already know: that visitors use the audio guides less as they progress through the exhibition. At the very least, it tells us that there are no guides where users need to use controls more because they are more difficult to hear.
Breaking audio control usage down into whether or not visitors use the QR codes does show some interesting differences. Mute and unmute functionality was provided so that screen readers could continue to be used when visitors stop an audio guide from playing. This graph shows negligible use by both QR and non-QR code users. Could this be due to the placement of the mute button on a different line from the speed controls?
The above graph appears to show QR code users using the play and pause controls more than non-QR code users. This is because there are simply more control usage events by QR code users in the sample (5869 compared to 597.) The same graph expressed as percentage of control use out of total control usage for QR and non-QR code users shows these groups as using the play and pause buttons with nearly the same frequency.
Looking at the usage of the speed controls, it appears that QR code users use these more frequently
but applying the same percentage approach shows that usage is fairly consistent with the exception of QR code user usage of speed 1.5x. This is interesting in that the 2x speed was provided on the hypothesis that hearing disabled visitors would prefer this speed and that the audio guide would be used by this group. Care should be taken in interpretation, however as the percentage difference is small (note that the axis is 0 - 10%.)
taceybadgerbrook
commented
2 years ago The Captions and transcripts page for In Plain Sight has attracted 328 users since the exhibition opened on 20 October to 27 November, almost a quarter of the number of users who visited the audio guides page.
taceybadgerbrook
commented
2 years ago The 46 visitors who visited the Captions and transcripts page via the QR Code in Gallery 1, 45 spent an average of 1 minute, 23 seconds reading the content readily available on the page. The Read full transcript button was clicked for only 6 of the 28 stops and for 4 of these 6, the content wasn't even read.
The Captions and transcripts page is very long and much of the material can be read without using the Read full transcript button, so it's easy to see why this was the most popular page (Note: the Hide full transcript button registers as a return visit to the Captions and transcripts page.)
It is not possible to analyse Read full transcript button usage for non-QR code users as this event is not tracked; however, the page is very long and it is unlikely to differ significantly from the QR code analysis. It would be interesting to monitor this page with Hotjar to see how far down the page users scroll. If some of the content was judged to be sufficiently short so as not to merit a Read full transcript button, could the user experience have been improved by offering only 1 Read full transcript button to allow users to scroll uninterrupted down the page? and maybe provide a Next transcript button to advance to the next stop?
taceybadgerbrook
commented
2 years ago The Watch BSL videos page has attracted 122 users since the exhibition opened on 20 October to 27 November. While this is the lowest number of users of any of our guides so far, the BSL videos will also appeal to a smaller number of users.
taceybadgerbrook
commented
2 years ago 20 visitors who accessed this page via the QR Code in Gallery 1. Only 5 of the 28 videos were clicked on and of the 5, the longest watch time was 8 seconds. This is very low take up and may be due to the fact that the BSL videos card will be bottom of the exhibition guides screen on a mobile device.
102 visitors not using the QR Code found their way onto the BSL videos page but it is not possible to tell whether or not they viewed any videos as this is not tracked.
jennpb
commented
2 years ago Summing up our conversation from UX Club:
I want to make sure the analysis covers everything from this Slack doc https://wellcome.slack.com/files/T0442CG7E/F04C8L0BZ60.
Understand how the In Plain Sight guides are currently being used, identifying any potential pain points.
This analysis may involve setting up additional tracking in order to get the data we need.
Besides the analysis, an output of this work should be a slide deck that communicates the findings so colleagues across WC can read and understand.
- Can we identify patterns of drop-off within an exhibition?
- Can we identify any performance issues that inhibit page loading/rendering of content in gallery?
- Are there differences in use depending on whether a person is using the guides remotely or in gallery?
- Are there differences in use depending on whether a person has accessed the guides via the exhibition page, or from QRcode?
In terms of language, if our tracking includes exhibition stops, we should refer to stops rather than pages
Next steps:
jennpb
commented
2 years ago @taceybadgerbrook Would you mind linking to the Word doc you mentioned, and making edits in that rather than as separate comments on this issue? It'll just help us to review and discuss.
taceybadgerbrook
commented
2 years ago Only 1 visitor who used the QR code in Gallery 1 went on to look at the Caption and transcripts and Audio guides pages for Gallery 2. Only 3 visitors who used the QR code in Gallery 2 went on to look at the Caption and transcripts and Audio guides pages for Gallery 1. None of the audio guides nor any of the BSL videos were played. 8 out of 652 or 7% of Audio guide pageviews for In Plain Sight were followed by clicks to change exhibition. 13 out of 925 or 8% of Captions and transcripts pageviews for In Plain Sight were followed by clicks to change exhibition.
On this basis, providing links to view guides for other exhibitions does not appear to encourage usage. Given that visitors can always go on to scan QR codes in other galleries, providing these links may actually be getting in the way of users navigating to guide content.
One interaction that doesn't appear to be catered for is switching between guides within an exhibition. For example, if an audio guide user wanted to read more about certain exhibits in Captions and transcripts, it's easy enough to go to the audio guide page and scroll down to go back to the guide. If however, they wanted to return to the Captions and transcripts page later in the exhibition, they would find it very time consuming trying to find the appropriate text or check whether there was even any additional material available. Offering a comprehensive audio/transcript/bsl set of links for each stop might increase usage. For those visitors who require one type of guide over the others, they can still use the page for the one guide.
taceybadgerbrook
commented
2 years ago Here is a link to the original analysis guide: https://docs.google.com/document/d/1H0y15MxidCFxy3bSwHMOCQ0AuEpEZFw6/edit?usp=drive_web&ouid=108386386272730213157&rtpof=true
taceybadgerbrook
commented
1 year ago These tickets have been created to try and develop the guides further:
We want to know if the following change has improved the user experience creation of exhibition guides
Definition of success is guides are used while exhibition is open guides are used after exhibition closes
We'll know if we're succeeding if we measure the following metric and this happens All guides are on one page, so analysis of GA event category Audio by label.