search

bitcoin-dot-org / Bitcoin.org

Bitcoin.org Website
https://bitcoin.org/
Other
1.57k stars 2.04k forks source link

Need way to compare wallets #2861

Closed crwatkins closed 5 years ago

crwatkins commented 5 years ago

We present a fairly good selection of wallets to choose from, but little guidance on the process of selecting a wallet. I suspect most users play whac-a-mole on the wallet listings going from wallet to wallet trying to differentiate them by reading our fairly accurate descriptions ("scoring"). As I (and others) have written in other issues and PRs I would like improve this situation in three stages:

  1. Expose existing wallet scoring information in a format that is easier make comparisons
  2. Add a "Good for new users" badge for wallets that are best for first time users
  3. Add a feature/capability chart for wallets (e.g. multisig, segwit, bech32, lightning, etc.)

I've opened this issue to discuss the first item; the other two will be the subject of subsequent issues and PRs. We've already had a fair amount of discussion in another issue that I originally opened almost a year ago to discuss specific bugs related to the wallet page redesign. I would like to move that discussion here to focus on this one issue with a title that is much more obvious to interested parties.

Background on existing scoring

Wallets are "scored" on six categories, one of which is privacy. Privacy has three subcategories which are similarly scored. The privacy score is manually (not in code) determined by the algorithm on the wallet criteria page based on the three privacy subcategory scores. Scores are not numbers, but rather are text labels which refer to two or more text descriptions of that category. Wallets are scored by choosing the text (and thus the label) that most accurately describes the wallet. All labels begin with the characters "check" followed by one of the four following sets of characters:

Currently on bitcoin.org, we indicate "fail" with an orange ball beside the text explaining the score while the other three ("good", "pass", and "neutral") are indicated with a green ball.

Proposal

My proposal is that we expose our existing scoring in a way that is easy for users to consume and compare. Except as I explained above, we don't currently expose the good/pass/fail/neutral levels. I further propose that we expose this visually.

Solicitation

I'm looking for ideas on how to be express this. Previously, @harding has suggested using Harvey Balls to augment potential color presentation of these levels, for example

              | Control | Validation | Transparency | Environment | Privacy | Fees
   Foo Wallet |    ●    |      ◕     |       ◔      |     ◔       |  ● ◕ ◔  |   ◕

In the past we've discussed a chart of all wallets, and we've also discussed adding these visual scores to the wallet icons on the selection pages. Some examples can be found by wading through the previous issue.

My Position

After considering this for almost a year now, my position is

  1. I would currently prefer a single page chart of scoring that puts the information as close together as possible to facilitate visual comparison. However I'm still looking for another good idea that will surprise me.
  2. I'm opposed to generating any new ratings or scoring or description in the context of this issue. This issue attempts to expedite the disclosure of our existing information to users, not create new information.
  3. We should not attempt to automate the selection process for the user with sorting, ranking, or numerical scoring. I don't believe that the granularity of our scoring nor the distribution of scores lends itself to that nor do I want to presume too much about the goals of the user. An obsolete chart of distribution can be seen here.
  4. The current good/pass/fail rankings are far from perfect, but we already have them, and we have tacit agreement on them, and as time goes on, I become more impressed at how well they were chosen. I'm hoping that we won't need to use those actual words, but rather express them visually.
maxwellmons commented 5 years ago

Hello @crwatkins, can you reopen and rename #2347 to the title of this issue?

People spent lots amount of time working together on #2347 to improve the wallet pages and you've closed it without asking anyone. It included different ideas about better ways to compare wallets.

This new issue essentially starts the conversation over without anybody else's ideas present.

crwatkins commented 5 years ago

@maxwellmons Thanks for your input but in a word no. I opened that issue to fix some bugs and those bugs are now fixed so it is appropriate to close it. You are welcome and encouraged to open your own issues and comment on any issues you like, even closed issues.

wbnns commented 5 years ago

Catching this issue up with current status and progress:

We are having progress meetings each Friday at 3PM UTC and can use this thread to communicate each week in the interim. All are welcome to contribute and share their feedback.

@natiwa is currently standing by until Friday so that anyone who would like to add feedback and share ideas has a chance to do so. On Friday we will review feedback that has accumulated and discuss next steps before further revising the different draft wireframes.

Cc: @natiwa, @Cobra-Bitcoin, @maxwellmons, @crwatkins

natiwa commented 5 years ago

@wbnns I will refer to this on Friday before our meeting.

wbnns commented 5 years ago

@natiwa Ok, thanks.

wbnns commented 5 years ago

@crwatkins Hey, a couple of questions before the meeting later this week:

Add a "Good for new users" badge for wallets that are best for first time users

As the wallet maintainer, which wallets do you recommend for this?

Add a feature/capability chart for wallets (e.g. multisig, segwit, bech32, lightning, etc.)

Could you please confirm a list of features/capabilities you recommend to appear in a chart?

crwatkins commented 5 years ago

@wbnns Thanks for asking! As I mentioned above

I've opened this issue to discuss the first item; the other two will be the subject of subsequent issues and PRs.

I didn't expect items 2 and 3 to be tackled right now because of the amount work that might be required. I had hoped to open separate issues for each of those and invite discussion and input from the community. I hope I will be pleasantly surprised, but I suspect it will actually be difficult to arrive at consensus on what wallets are good for first timers, let alone the criteria. It's easy for me to give a small list that might qualify, but everyone has their own favorite and may be insulted that it is not on the list. That said, I recently performed such an exercise and have five quasi-criteria (some of which are extremely subjective and thus hard to explain) which I used to determine my own list. Since you asked:

My example list of features/capabilities from above (multisig, segwit, bech32, lightning) plus hardware wallet support, full node support, and coin shuffling, might be good to start with. We should solicit input on other suggestions.

wbnns commented 5 years ago

@crwatkins

Ok, understood, thanks. Do you have any other thoughts or feedback on the new work that @natiwa and team have done so far that we can review at the meeting tomorrow?

crwatkins commented 5 years ago

Thanks @natiwa. I think @wbnns has a lot of good comments on the new designs. I'll throw out a few random thoughts and things to keep in mind.

First, as I've mentioned before, I'm not sure we have to provide active filters to filter scores. That said, if they are done right and the actual filters are optional (as I think they are in all three designs), I don't think there is much downside (I'll mention potential downsides below). However, I think it is important to keep in mind that we have no OS category with a dozen (or more) wallets in it. We probably should anticipate some growth, but the number of listed wallets has remained surprisingly stable over the last four years. That said, I think we could avoid having to implement score filters if we wanted to and simply provide a visual comparison of the less than a dozen wallets selected. It could make our work a lot simpler and potentially end up with a simpler and easier to understand UX.

@wbnns pointed out some challenges. I'll just mention a few, more as things to think about rather than a comprehensive list of issues to be addressed.

  1. I'm not sure how we are going to adequately describe our score categories to users, particularly new users if we do have an active filter interface.

The only thing I've been able to come up with is a big long list of phrases to choose from. For example:

Which best describes the wallet you are looking for?

It would be accurate but not very reasonable.

  1. I also think there may be some filtering that would create awkward results. For example if you said environment is important, but you are looking for a desktop wallet, will we provide no results? (they all "fail")

  2. Also be aware that two categories do not have any "good" entries.

  3. Likewise many of the categories don't apply ("neutral") to hardware wallets.

  4. I'm afraid that the term "fail" is wrong. While we think of the category as fail, we still list the wallet, so in that regard, it's not really a fail. Perhaps if we need a word to expose to describe that category. It might be "poor".

wbnns commented 5 years ago

@crwatkins Thanks a bunch for putting this feedback together.

natiwa commented 5 years ago

At first thanks @crwatkins and @wbnns for the great feedback.

I have a New idea. The flow can be as follow:

  1. User click “Choose your wallet” button on the home page
  2. The page “Choose your wallet” appears with the “wizard” and a “skip a wizard” options
  3. If he chooses the “wizard” option: User step by step select options and answers to questions about:
    • Operating system (we can present a pros and cons of each OS)
    • Level of advancement
    • Needs and preferences
  4. A user will get the result based on the selected criteria,
  5. If user click “skip wizard” option he will be directly moved to the page with options/ filters on the left and all wallets preview

---------> Let me prepare the wireframe of the wizard flow. That would probably explain a lot and make things way more clear.

@wbnns Answering your questions:

  1. As I mentioned, we can have the “I’m an Advanced user” selected by default, and we can know by code what type of an operating system user has. Screenshot 2019-03-15 at 12 23 18
  2. The order should be alphabetical in my opinion. I think only wallets that have a "good" result of our selected criteria should be displayed. As an example: If I selected "privacy" and "control" I will see only wallets that have Good Privacy and Good Control. If I do not select any criteria I will see all the wallets.
    1. I did it on purpose. I was thinking that a new user has no idea about those criteria, but now I know that we should have them by better described for the new users. Please see the NEW FLOW at the top of my message.
    2. I thought we shouldn't show more that one wallet for a new user. More than one would be a paradox of choice for them. That's why I put "We highly recommend..." and show one wallet. Now I think "new flow" and the wizard will change everything.
    3. I agree. But doesn't "fail" a bad thing?

@crwatkins Answering your questions:

  1. Then we should inform the user about the result and explain what does it mean and why is it.
  2. Which categories?
  3. Ok. What we should do in that case?
  4. We can, of course, figure out the correct naming any time.

All the best, Natalia

natiwa commented 5 years ago

I'm afraid the google calendar again confused something with time zones. In my google calendar, I have an invitation to a 4 pm CET, not 3 pm CET. Anyway, I'm right now.

wbnns commented 5 years ago

@natiwa Hello, apologies if there was some confusion. It is scheduled for 3PM UTC, which is 4PM CET. Are you still available at that time? Alternatively, if more convenient, we can reschedule to Monday, and review the new idea you mentioned in the interim.

natiwa commented 5 years ago

Yes, that would be actually great. I will have new wireframe ready by then. Can we reschedule it once on Monday? Does 3 PM CET works for you?

wbnns commented 5 years ago

@natiwa Ok, sure. Will look forward to connecting on Monday at 3PM CET (2PM UTC).

natiwa commented 5 years ago

@wbnns Thank you, talk to you then and have a great weekend.

crwatkins commented 5 years ago

@natiwa To answer your category questions above, you can refer to the chart that I did about a year ago. Some wallets have come and gone and maybe a score has changed, but I'm not sure that matters as it still gives us an idea of what the distributions look like.

natiwa commented 5 years ago

Hello all,

Please find the new idea here: Clickable mockup An alternative version of the Harvey balls : Link When user click skip helper he will see all wallets: Link

See you on the call.

Cobra-Bitcoin commented 5 years ago

I really like that new idea. By default the UI could assume the user is a beginner, since most visitors to that page are, and have the checkboxes pre-selected to match their needs. This means some wallets will have some natural advantage, but I really think newbies should be spared from accidentally ending up using a wallet like Wasabi or Eclair which won't work well for them.

I think with the ordering we shouldn't have it be alphabetical, we can either have it randomly ordered or add a new score to each wallet, something like a confidence score which represents our confidence in a wallet relative to other wallets. But I prefer random.

crwatkins commented 5 years ago

But I prefer random.

I prefer random. I think the alternative would be way too hard to arrive at consensus on and and to generate new scores, and of course, I tend to prefer the lazier simple route.

crwatkins commented 5 years ago

My following comments apply to the Hardware Wallet section (only).

I'm wondering if we should treat hardware wallets differently than the other wallets in regards to displaying and filtering since they are so different. Currently, our hardware wallets

Much of this stems from the fact that our hardware wallets are not full wallet environments, but perhaps more accurately described as signing engines. A user currently is not going to choose a hardware wallet without a companion software wallet, so some of the categories don't make sense. I think we can

  1. Continue as before and treat them the same as before marking some scoring as neutral
  2. Continue as before, but remove the score (and filter) from some categories (and not display the category)
  3. Treat them as a separate group

I'm leaning toward 3 because because I can't use a hardware wallet standalone; I need a hardware wallet plus a software wallet. It may be just too awkward to continue to treat them the same.

Thoughts?

wbnns commented 5 years ago

@natiwa Thank you all for getting together on Monday to discuss the latest progress. Can you also please do a version/flow where the user does the following:

  1. Starts off the flow by being presented with a page where they identify themself as a new or experienced bitcoin user.
  2. If new, the user would then go to a page that is structured similarly to the You Need to Know page, where the different wallet criteria are explained, followed by
  3. A page where the user selects what wallet environment he/she would like (i.e. web, mobile, desktop, hardware), followed by
  4. A table of wallets for that environment along with the scoring criteria/colored dots

If the user indicated that he/she was an experienced user in step 1 above, step 2 would be skipped and he/she would go directly to step 3.

@crwatkins @Cobra-Bitcoin Ok, we will plan to randomize the table.

@crwatkins Regarding the hardware wallets, we can also do a couple wireframes to see what that might look like, too, as we get some options for these new flows mocked up.

natiwa commented 5 years ago

Hey, Thanks for your feedback.

Some of my thoughts/questions :

If new, the user would then go to a page that is structured similarly to the You Need to Know page, where the different wallet criteria are explained, followed by

  • Shall I take all the info from the "You Need to Know" page? Should I add anything more?

I will prepare everything on Monday. Natalia

wbnns commented 5 years ago

@natiwa

Thanks, regarding:

Do advanced users need a wizard? They can skip it and go directly to that view Link

No, and this would be a new wireframe, where only the table is shown (Step 4, above).

Shall I take all the info from the "You Need to Know" page? Should I add anything more?

No, this would be a new page, laid out the same way as the "You Need to Know" page, with different content. The content would be the scoring criteria along with their explanations.

What about that step?

This is a different flow; this step wouldn't exist. It's its own version that we can compare to the previous flow that you put together on Monday, as one of the options that the community will eventually vote on.

Let's connect first to go over this before you put together the wireframes.

natiwa commented 5 years ago

Hi Will,

Please take a look at this. I hope this is what you have in mind. Link to the clickable wireframe

Natalia

Cobra-Bitcoin commented 5 years ago

@natiwa I really like that wireframe, looks very nice.

wbnns commented 5 years ago

@natiwa

Thank you very much. Yes, great job translating the notes into the new wireframes.

Let's plan to review all wireframes to date at this Friday's meeting with the goal of beginning the actual mock-ups (handheld/tablet, desktop) on Monday.

We'll be leaving some additional comments in the Invision previews in the interim.

wbnns commented 5 years ago

Hello, this is just a quick update to all who are following this thread to confirm that the wireframes we've previously been working on, are now being converted to actual mock-ups (desktop and mobile) for everyone to review.

natiwa commented 5 years ago

I will send all previews till Wednesday. Can we once move our Friday meeting to Thursday at the same time?

wbnns commented 5 years ago

@natiwa

Thank you. Let's plan to meet on Monday instead of Thursday, that way we can give people a chance to leave feedback on the new previews, prior to the meeting.

Cobra-Bitcoin commented 5 years ago

Very excited to see the mock ups :)

natiwa commented 5 years ago

Hi all,

Please take a look at all mockups:

Flow 1: Desktop Tablet Mobile

Flow 2: Desktop Tablet Mobile

You are able to comment directly in InVision.

wbnns commented 5 years ago

@natiwa

Ok, great, thank you - we'll review.

Cobra-Bitcoin commented 5 years ago

Flow 2 is absolutely beautiful! Completely blown away by how awesome that is, seriously cool. I love the interactivity of the helper and how it pushes the user to exactly what they need. That's 100% what we need to be doing. Flow 1 is much too basic, and users really need more of a gentle hand and interactive process. I'll leave more detailed feedback later.

Cobra-Bitcoin commented 5 years ago

I've left feedback on the mockups for flow 2. There seem to be a few issues with fitting the comparison table properly into devices with smaller screens. But for the most part, it's hard to find flaws or other issues (except for a few other small nitpicks).

Ideally in the live site we will animate these transitions in the wizard. Looking forward to seeing more progress on this.

wbnns commented 5 years ago

@Cobra-Bitcoin Thank you.

@crwatkins Could you please review the mockups and add your comments as well?

crwatkins commented 5 years ago

Here are some scattered comments on the mockups, which are mainly operation/implementation considerations and not look and feel. Some I may have mentioned before in this or other threads, and I apologize for the duplication.

  1. I think it is probably time to drop Windows Mobile (since Microsoft has). I mention that now in case it affects layouts.

  2. We will need some criteria to decide what wallets are good for new users. @wbnns asked me for a sample set which I provided, but it may not be as easy as it sounds to come up with concrete criteria and I solicit input from the community. I definitely want to avoid as much subjective categories as possible that tend to make reviews more difficult and contentious.

  3. Is it anticipated that the expert selection would include all wallets while the novice selection would include a smaller subset? (That would probably be my preference.)

  4. In some of the mockups there is a "basic knowledge" selection. With the number of wallets we have listed, I think it might be even more difficult to form three groups than two.

  5. Even with two knowledge levels, and criteria filters, we may have some empty results. We may have to design some flows to guide users in expanding their criteria.

  6. I like the color balls, but I suspect we also need some shape or word clues for color disabilities? Do we have to combine the mockup proposals for that?

  7. As I mentioned above, the hardware listings might be challenge to display in a meaningful manner given very few differentiating criteria.

  8. The Good/Pass/Fail perhaps could be renamed. We have never actually exposed these words to end users yet. I'm very cautious on making this suggestion however, because I continue to discover over time how brilliantly the scoring was designed and chosen by @saivann, @harding, and others. Input from them on the wisdom of potentially using other labels to describe these scores (e.g. perhaps "Poor" instead of "Fail") would be greatly appreciated.

wbnns commented 5 years ago

@crwatkins Thank you. Regarding your comments:

  1. I think it is probably time to drop Windows Mobile (since Microsoft has). I mention that now in case it affects layouts.

Agreed. Thanks for mentioning. Discussed this with Natalie during our meeting last Friday. Will open a PR to remove.

  1. We will need some criteria to decide what wallets are good for new users. @wbnns asked me for a sample set which I provided, but it may not be as easy as it sounds to come up with concrete criteria and I solicit input from the community. I definitely want to avoid as much subjective categories as possible that tend to make reviews more difficult and contentious.

👍

  1. Is it anticipated that the expert selection would include all wallets while the novice selection would include a smaller subset? (That would probably be my preference.)

Yes, this is what we were thinking.

  1. In some of the mockups there is a "basic knowledge" selection. With the number of wallets we have listed, I think it might be even more difficult to form three groups than two.

Agreed, we're going to update the mockup to only show two groups.

  1. Even with two knowledge levels, and criteria filters, we may have some empty results. We may have to design some flows to guide users in expanding their criteria.

Yes, we're going to organize the different selections in a spreadsheet so we can start to see how these will filter down to get a better idea of how practical and/or helpful some of these filtering options are going to be.

  1. I like the color balls, but I suspect we also need some shape or word clues for color disabilities? Do we have to combine the mockup proposals for that?

Not sure; good idea to use shape and/or word clues.

  1. As I mentioned above, the hardware listings might be challenge to display in a meaningful manner given very few differentiating criteria.

Yes. We're thinking we can take another pass at this once some of these other items are in order.

  1. The Good/Pass/Fail perhaps could be renamed. We have never actually exposed these words to end users yet.

How about: Favorable or Good (Green circle) / Caution (Yellow triangle) / Warning (Red square) - ?

We're not sure the word "Fail" should be used anywhere, as we suspect most people will have an inclination to not want to use products and/or services with something failing associated with it. We were thinking it might be better to use the word "Warning", instead.

Also, we're suggesting the word "Caution" instead of "Pass" in association with the yellow score - the color yellow could be confusing, since in many cases yellow is generally associated with slowing down, yielding, or exercising caution.

natiwa commented 5 years ago

Hi all,

I have improved all your comments from Invision.

You can find improved mockups here: Flow 1: Desktop Tablet Mobile

Flow 2: Desktop Tablet Mobile

Natalia

Cobra-Bitcoin commented 5 years ago

"Good, Caution and Warning" is pretty nice. Though we can always change around these words after the redesign, probably shouldn't think too much on them right now.

The new mockups look pretty much perfect, seems like we're close to begin implementing now? I'm still 100% behind flow 2 and on having the wizard/helper. Is flow 1 still being considered? To me flow 2 just seems so much superior, and has clearly had more time and effort put into it. When it comes to implementing though, we're going to have to test it pretty thoroughly since there's a lot of moving parts involved across many different device sizes.

I also really love the "Discover more" section floating to the right of the content, very useful.

natiwa commented 5 years ago

I'm glad to hear that. Let's wait for the other feedback but I feel we are very close to finding "that one" solution.

wbnns commented 5 years ago

@crwatkins

Hey, for the Wallet type comparison page:

image

Do you have an initial set of positives and negatives you would suggest to include under each wallet type?

wbnns commented 5 years ago

@Cobra-Bitcoin

Yes, it's getting closer to coming together. Flow 2 seems to be the front-runner. We've kept Flow 1 for reference if we need to fall-back onto anything more simple in case any of these enhancements turn out to not be feasible. Flow 2 is getting much more attention between the two at this point.

In Flow 2, the table layout with the scoring still needs some work, mostly on handheld/mobile, and we're trying to see if there is another way we can present a comparison of scores without a table or an awkward horizontal scroll. In current form, it may be difficult to read on a smaller handheld screen, and on some devices, due to the number of scoring fields, some of the longer scoring terms (i.e. Environment, Validation, Transparency) run the risk of overlapping onto other scoring terms.

We have a progress meeting scheduled for Monday at 14:00 UTC.

Related: #1148, #2892, #2858, #1986, #2723

wbnns commented 5 years ago

We've prepared tables of the current wallet scores for comparison --

Desktop (Linux, Mac and Windows - scores and available wallets are the same across all operating systems):

image

Android:

image

iOS:

image

Hardware:

image

Web:

image

...in conjunction with examining what the user experience might be if using the new criteria selection tool in Flow 2:

image

Some notes / observations:

Please let us know if anyone has any feedback they would like to add.

Cc: @natiwa, @Cobra-Bitcoin, @crwatkins, @maxwellmons

wbnns commented 5 years ago

Apologies and an additional FYI that the previous message has been edited to include the scoring tables and wallet availability for all operating systems / environments.

Cobra-Bitcoin commented 5 years ago

Perhaps we can have a "Do you intend to store a lot of Bitcoin?" step, and if they answer yes, we just send them straight to hardware wallets. IMO most newbies are better off using a hardware wallet for anything >$1000.

crwatkins commented 5 years ago

@Cobra-Bitcoin I think a "purpose" selector is a reasonable consideration. @JTuwiner's wallet selection page (which we indirectly link to from our exchange page and our resource page) provides a good example of this.

natiwa commented 5 years ago

@wbnns Have you seen this solution for a mobile device? LINK When you select the one criteria from the list you would see the comparison of wallets for only that criteria.

crwatkins commented 5 years ago

@wbnns Thanks for the tables above

It appears that at present on bitcoin.org wallet pages, red dots are never shown despite failing scores. We need to clarify exactly when yellow dots and/or red dots should be shown.

Could you show us the table with the "red dots" also?

On the subject of the "warning" and "caution" terminology: I ran those words by a few people and I got the the response that those two words basically mean the same thing and no one knew for sure which one was better. I think we might have to use something different.

wbnns commented 5 years ago

@natiwa Thanks for the link, yes, looks great. Can you please work that in for the other places where a table might have been shown on mobile/handheld devices to see if that solves remaining viewport issues?

@Cobra-Bitcoin @crwatkins We can add some kind of question to the wizard to direct people directly to hardware wallets - not sure we should denominate it as $1000 US Dollars, though. In reviewing traffic for the past 12 months, out of 11 million visitors to the site, 80% were from outside the US. Perhaps we could generalize the question to not tie it to one nation's currency equivalent, especially since so many people are coming from outside the US; but not sure how to word it objectively (e.g. "Do you plan to store a large amount of bitcoin in your wallet?" is subjective).

@crwatkins I've updated the above table to indicate where red would be present. As for terminology, understood, thanks - when we have some live previews available, we can come up with a few options and then we can do a user poll. In the interim, I've renamed the fields in the table to good (checkgood), pass (checkpass) and fail (checkfail) to correspond to the values in the data fields that previously weren't exposed on the wallet pages. On the subject of the scoring values: