Creator Tokens — Viewer: Testing

[toiletgranny]

This phase can include activities such as conducting moderated/unmoderated usability tests on prototypes from the LoFi phase, doing surveys, or other types of user research.

The expected outcome is a brief summary of key findings, a list of recommendations, and a list of test participants to be paid out (if applicable). Learn more from Approaching testing in Atlas Notion article (WIP).

Depending on the results, the next step is either proceeding to the next phase of the design process or going back to the LoFi phase and repeating the tests if needed.

[KubaMikolajczyk]

I updated the designs & prepared quite a few elements to test in usability test - I talk about all of this in the loom update below 🎥 Video - https://www.loom.com/share/dc00203209574d9e8ca560731a983bda 📄 Figma - https://www.figma.com/file/VzPBMpUnGjGOHMSW2ZHbGY/Creator-tokens?node-id=2294%3A194530

[bedeho]

Really great work, very eager to see the results of those tests, I have two suggestions for copy changes

• Stake: This is too abstract, can't we here do something like Claim X $JOY, making it clear how much money they can get for pressing this button based on their share of this token, because this button can only be shown for token holders anyway.
• Whitelisted: This terms is really something that describes a status of someone, like say a member, in some context where there is a whitelist. So the natural interpretation of this label is that the viewer is in fact on the whitelist, which is not the intended message. I think we should instead use label Whitelist, or Whitelist Only or something, and then a distinct signal is needed to inform the member whether they are in fact present on the whitelist of this token or not, which currently is missing?

[KubaMikolajczyk]

@bedeho thank you for the feedback and I think that during the test we confirmed your suggestions but more about that below 😅

CRT Viewer usability tests

Overview

We conducted 5 usability tests for the CRT Viewer experience. We hoped to gain a deeper understanding of potential user pain points and preferences.

👥 5 participants

• 2 users from the Deso platform who were with us on the last CRT test
• 3 Joystream users and Creators with at least a basic understanding of what CRT is

Joystream members to be paid (tJoys) 👈 @bwhm

5DX9Vvk4uP8YNFSsvWGbjBcTG21PJbeLpgTRgCVpGQg2bDSx 5Dh8Lw11DPxmVE5GsFFoBvBHU7XFkdoLh2MzuoK5Sh6r9Aws 5CrD4RtGauzbpsypoG8yEoxxkuLusb25nCfgQxNCi4d4ZSV1

Deso members to be paid (Bitcoin Cash) 👈 @bwhm

qpaqss6dtelkq9x35gks6r54g3xktareauz4dnasel QQTYUTSND4WSVJY5X9X5TNQA4FXZKRTCEYX2WEGYPG

What do we want to know?

1. Which layout (two-column or one column) is preferred by the users, which is easier to read? Which performs better when there is little to no content?
2. What is the most crucial information they are looking for when they open a token page?
3. Do the users understand the overall information presented on the token page?
4. Do people understand token verification and if it is important to them?
5. Do people understand statistics and do they know what those pills with percentages represent?
6. Do people understand the holders' table? Do they know what percentages in the table mean?
7. How many holders should be displayed for the user upfront on the token page?
8. Do they understand all 6 presented states of the token?
9. Do they understand the difference between AMM and Public sale
10. Do they understand when they are on the whitelist and when they are not?
11. Do they understand what stake button on an active revenue split state does?
12. Are there any issues with the purchase flow?

Research outcome

1️⃣ Which layout (two-column or one column) is preferred by the users, which is easier to read? Which performs better when there is little to no content?

To test that I showed all participants 2 different layouts one with two columns and the second one with just one column. Both of that layouts had 3 variants - one with a lot of content, one with only a few elements, and one empty. 4/5 of participants preferred 2 column layout when it was full of content, 5/5 preferred 2 column layout when there was only a little content provided, and even 4/5 preferred 2 column layout when the creator didn't provide any content at all. 2 column layout seems to be a clear winner with such opinions as: "faster to scan; visually more attractive; less scrolling".

2️⃣ What is the most crucial information they are looking for when they open a token page?

Most users pointed price to be the first thing they look for, then the second place would be token supply, and shortly after that token description (intro, benefits, about) closing the top 3. ⭐️ I will add token supply to the same line as price in the price widget just as Dmitry suggested, tests show us that it's needed.

3️⃣ Do the users understand the overall information presented on the token page?

Yes, they do. They scan it easily - usually starting from the prominent intro video and then moving to explore more info about the token itself in the right column. There is a problem with understanding such concepts as Royalties split (users don't know what are the revenue streams and they lack additional info about revenue splits). But overall even if a user said that the concept is not clear then when asked about their best guess - they almost always guessed correctly. + People do read information tooltips and learn :)

4️⃣ Do people understand token verification and if it is important to them?

Yes, they understand and all of them rated such information as very important for them.

5️⃣ Do people understand statistics and do they know what those pills with percentages represent?

People do understand the statistics, we should add a market cap to the statistics as well as that is something that users were looking for there. Pills with percentages confused almost all of the users because they didn't know over what time is this change (so the tooltip with information here didn't work 😅). 3/5 of users said that they would appreciate the possibility to choose the range of the change that is displayed for them in statistics.

6️⃣ Do people understand the holders' table? Do they know what percentages in the table mean?

People do understand most of the informations in the table, although 2/5 users didn't know what is vesting and wanted to learn more about that as they weren't able to guess correctly. Almost all users knew that the percentage represent the % of token supply.

7️⃣ How many holders should be displayed for the user upfront on the token page?

2/5 users suggested that 10 is the only right number to be displayed upfront, 2/5 suggested that they like 10 but they would appreciate even more 20 or 25, 1/5 said that 10 or 8 would be ok, but not less. ⭐️ Based on those suggestion I would recommend that we display 10 holders on the token page and the rest after user clicks show more button.

8️⃣ Do they understand all 6 presented states of the token?

They do understand the states and had no problem identifying in what state the token currently is.

9️⃣ Do they understand the difference between AMM and Public sale

✅ Good thing is that all testers notice the AMM / Public sale states on their own and they could tell that there is the difference. They also read the tooltips of AMM and Public sale and when reading the tooltips they understand the idea. 🛑 Bad thing is that there is still confusion - naming of public sale inclines that it is indeed public and when users see whitelisted token they dont know why there is a "public" sale. ⛔️🆘 Even users who knew before what AMM is, were disappointed when they find out that both creator & holders are not going to receive any revenue % from AMM token purchases. One user mentioned: "It's not like everywhere else, but ok". I know that including % of revenue in AMM was taken out of scope but it's at least worth inspecting more carefully later.

1️⃣0️⃣ Do they understand when they are on the whitelist and when they are not?

Yes, they do. 100% of users guessed correctly both when they weren't on a whitelist and also when they were on it. ⭐️ One minor improvement can be done - just as @bedeho mentioned in feedback above - whitelisted - written as a token type can be misleading for users - giving them a false idea whether they are on a whitelist so we should use a different naming as suggested above 👍 ⭐️ Also the content of "you're not on whitelist" state can be improved upon feedback from the users. Instead mentioning paused sales we can just explain in the banner title that all staking users are now being paid as this is the most important information.

1️⃣1️⃣ Do they understand what stake button on an active revenue split state does?

Almost all of them (4/5) understood that with stake button they can get access to their revenue share - 1/5 didn't know the staking mechanism and it was a mystery for him.

Even though vast majority understood the concept - few of them had problems with identifying how the staking will work (are they going to stake for some time? Or do they stake after the revenue split ends?) so probably a link where they can learn more about the whole process would be needed. When asked about how would they look for this information most answered: "either with the learn more link, or if its not there I would look for FAQ section (and started browsing in the left navigation panel).

⭐️ I would rename the stake button just as bedeho suggested above to "Claim X JOY" to get rid of possible confusion - and also include a info link to learn more about the revenue splits and staking.

1️⃣2️⃣ Are there any issues with the purchase flow?

✅ There are no issues. All participants were able to buy the token. 2/5 Users who didn't know about vesting (didn't know what it is) had to learn it from information tooltips. And they understood the concept correctly that they don't get all of the funds at once but rather gradually over time. But they mentioned that they would appreciate a link to some learning resources to learn more about vesting.

Additional thoughts and ideas 💡

1️⃣ FAQ

We are adding more and more complexity to the platform and I can see a growing need for some internal learning resource. It would be worth considering to start developing an FAQ for such elements as: creator token, revenue split, revenue stream (ways to earn), patronage, how to verify your token, amm, public sale, vesting, cliff, whitelist, transaction fee, platform fee and more. We could start with using Notion as a MVP and link to it from the platform and later on develop something included in Atlas.

2️⃣ Accesability

One of our testers had vision impairment and wore glasses.He was not able to identify our information icon as "information" icon - it looked for him like number 0 - so probably we could rework icon to better represent the state - that would be an easy fix for that. But when I told him that those are triggering tooltips where he can learn more - he didn't open them much because "probably" the text was too small for him to read - I would advice that at some point we should open accesability topic and care for all Joystream users. ⭐️ I would love to make some kind of quantity test on our Joystream users and ask them if they have any kind of disabilities - this way we could measure what % of users do we talk about :)

3️⃣ History of payouts

Some user asked for seeing the history of revenue splits so they can see when and how much the creator shared with holders - this may be worth to consider.

4️⃣ Joy token icon

Users didn't know in what currency we display the prices - and the Joy icon didn't ring a bell for them ( not surprising at all as they didn't see it anywhere before ) - but because they didn't know what currency it is they were confused a bit. We should at least add an information in a tooltip when user is hovering the price that this is indeed "JOY" currency.

5️⃣ Vesting time (and different vesting schedules)

Currently when users are inspecting the holders table on the token page they can see a total number of vested tokens but they dont know for how long they're vested and how many different vesting periods there are - so to account for that I propose that we would add an expandable section for each coin in portfolio page (which is not yet designed because its in the holders experience) - and there when users inspect particular coin in portfolio - they would see each individual vesting schedule. I will be exploring this approach when we reach hi-fi work for holder experience.

6️⃣ Relation between Viewer and this token holders

One of the experienced users wanted to see while inspecting holders table if among this token holders are people who own his own token and does he own any of their tokens. We can easily add this information in the tooltip that shows holders portfolio to give even more important context for viewers.

7️⃣ Naming of actions in AMM

One user was confused about the naming of the button "buy" in AMM as he thought that with AMM you mint (which is correct) and not buy. Should we change the action to Mint?

8️⃣ Soft-lock with holding CRT

One user was 🛑 terrified - by the idea that if there is no active AMM - he won't be able to sell his token: "And there’s no active sale, but does it mean that holders can’t sell it back? But I would not like to be forced to hold it, it wouldn’t be nice.". We could at least talk about it and think of solution.

Thanks for reading if you reached the end 🙇 All feedback highly appreciated, but I will read it after 10.07 as I just started my vacations 🌴

[bedeho]

Phenomenal work, really well done ;)

[dmtrjsg]

Testers paid out ✅

[dmtrjsg]

@KubaMikolajczyk amazing work 🤩, adding my 5p ahead of next review:

• Revenue split history> to my understanding that would be hard to find a compact way to display it in a reliably comparable manner, but we could probably show total channel revenue next to the Revenue share %, slightly compromising on immutable / dynamic variables for the benefit of having contextual info close to key indicator.
• Statistics > I think we need an alternative title here, but cannot quite figure it out. "Token economics", "Revenue details" these seem not much better. I also found myself thinking is it worth exploring putting it above the "Buy" widget; so that user interested in revenue details would not need to jump over this widget to view key info together with the contents of statistics. Mindful tho it may send the buy widget beneath the fold, which may be a worse-off trade.
• Public sale > thinking "Creator sale" could be an alternative, and using "Open" "Whitelist only" labels for wl param?
• Staking > If I remember correctly CRT owner would need to claim the attributed revenue split, so using "Claim" may need to be reserved for the "recovery" of staked tokens with premium. Meaning we could just go with "Earn XYZ by staking your CRTs now." ⚠️ this needs validation based on the runtime, as I am not 100 sure at the moment of writing this comment.
• Relation to owners > Interesting idea, would be good to explore further subject to tech constraints. Def makes product and coin discovery more engaging. ⭐
• Mint instead of Buy -> that one is tough.. my hypothesis is that we are more likely to get better CR retaining buy, and adding a small undercopy into the purchase flow somewhere along the lines of ""By clicking the BUY button you will mint XYZ in exchange for .. ", to fend off judgement from crypto veterans; rather than the opposite approach.