Need way to compare wallets

[crwatkins]

We present a fairly good selection of wallets to choose from, but little guidance on the process of selecting a wallet. I suspect most users play whac-a-mole on the wallet listings going from wallet to wallet trying to differentiate them by reading our fairly accurate descriptions ("scoring"). As I (and others) have written in other issues and PRs I would like improve this situation in three stages:

1. Expose existing wallet scoring information in a format that is easier make comparisons
2. Add a "Good for new users" badge for wallets that are best for first time users
3. Add a feature/capability chart for wallets (e.g. multisig, segwit, bech32, lightning, etc.)

I've opened this issue to discuss the first item; the other two will be the subject of subsequent issues and PRs. We've already had a fair amount of discussion in another issue that I originally opened almost a year ago to discuss specific bugs related to the wallet page redesign. I would like to move that discussion here to focus on this one issue with a title that is much more obvious to interested parties.

Background on existing scoring

Wallets are "scored" on six categories, one of which is privacy. Privacy has three subcategories which are similarly scored. The privacy score is manually (not in code) determined by the algorithm on the wallet criteria page based on the three privacy subcategory scores. Scores are not numbers, but rather are text labels which refer to two or more text descriptions of that category. Wallets are scored by choosing the text (and thus the label) that most accurately describes the wallet. All labels begin with the characters "check" followed by one of the four following sets of characters:

• good
• pass
• fail
• neutral

Currently on bitcoin.org, we indicate "fail" with an orange ball beside the text explaining the score while the other three ("good", "pass", and "neutral") are indicated with a green ball.

Proposal

My proposal is that we expose our existing scoring in a way that is easy for users to consume and compare. Except as I explained above, we don't currently expose the good/pass/fail/neutral levels. I further propose that we expose this visually.

Solicitation

I'm looking for ideas on how to be express this. Previously, @harding has suggested using Harvey Balls to augment potential color presentation of these levels, for example

              | Control | Validation | Transparency | Environment | Privacy | Fees
   Foo Wallet |    ●    |      ◕     |       ◔      |     ◔       |  ● ◕ ◔  |   ◕

In the past we've discussed a chart of all wallets, and we've also discussed adding these visual scores to the wallet icons on the selection pages. Some examples can be found by wading through the previous issue.

My Position

After considering this for almost a year now, my position is

1. I would currently prefer a single page chart of scoring that puts the information as close together as possible to facilitate visual comparison. However I'm still looking for another good idea that will surprise me.
2. I'm opposed to generating any new ratings or scoring or description in the context of this issue. This issue attempts to expedite the disclosure of our existing information to users, not create new information.
3. We should not attempt to automate the selection process for the user with sorting, ranking, or numerical scoring. I don't believe that the granularity of our scoring nor the distribution of scores lends itself to that nor do I want to presume too much about the goals of the user. An obsolete chart of distribution can be seen here.
4. The current good/pass/fail rankings are far from perfect, but we already have them, and we have tacit agreement on them, and as time goes on, I become more impressed at how well they were chosen. I'm hoping that we won't need to use those actual words, but rather express them visually.

[natiwa]

I have improved all your comments (The links are the same as previous) + created this new screen .

[wbnns]

@natiwa Ok, thanks, we'll review. 👍

[wbnns]

@crwatkins

Hey, just wanted to follow-up - did you get a chance to review the updated scoring tables and the additional observations/assumptions regarding the scoring specification?

We were hoping to get your feedback.

[crwatkins]

We can add some kind of question to the wizard to direct people directly to hardware wallets [...] not sure how to word it objectively (e.g. "Do you plan to store a large amount of bitcoin in your wallet?" is subjective)

Perhaps the criteria is "long term storage" vs "frequent small transactions"

[crwatkins]

• It appears 'checkgood' is never applied to any of the privacy subcategories, only 'checkpass' or 'checkfail' are used in association with scoring values. Also, there are instances where wallets receive a 'checkfail' in 2 out of 3 privacy subcategories but still receive a 'checkpass' (e.g. Bitcoin Wallet and Edge Wallet) for their overall privacy score along with the color green on the wallet page. Not sure it should be 'checkpass' if 2/3 of the subcategories are 'checkfail' - or that it should be green. For now, changing the Privacy column to TBD for wallets until we can get clarification on what the intended logic is (or what it should be).

This seems correct per our wallet scoring criteria for privacy which states:

To get a good score, the wallet must avoid address reuse by using a new change address for each transaction, avoid disclosing information to peers or central servers and be compatible with Tor.

To get a passing score, the wallet must avoid address reuse by using a new change address for each transaction.

That said, I've been concerned about the lack of differentiation of our privacy scoring for some time now, especially anticipating the day we would expose them to users in a more obvious presentation. For just this reason, a year ago, I proposed changing the privacy scoring in #2462 but failed to come up with a change that was acceptable (even to myself).

To further complicate this issue, I'll point out that the address reuse scoring is probably no longer pertinent since avoiding address reuse is now a requirement for listing #1327 (further complication is that we have a wallet listed as failing this, which probably isn't up to date).

I think it would be great if we could present this existing scoring in a way that the lack of differentiation doesn't confuse users. If we are unable to do that, perhaps we could leave the confusing criteria out of the selectors.

We absolutely need to upgrade this section of the scoring in the future, but I hope we can find a presentation for our existing scoring now.

[crwatkins]

• For hardware wallets, a 'checkneutral' term is used, instead of 'checkgood', 'checkpass', or 'checkfail' like in the other categories. [...] Also, this is appearing as green even though the term checkneutral is used, so we're not sure what the color should be.

I agree that we have very little differentiation for hardware wallets (that's not necessarily a bad thing), so we may not even choose to expose it with selectors. If we do, and we expose the checkneutral categories, I would suggest gray.

• We assume that if checkgood is present, the scoring color should be green, if checkpass is present, the scoring color should be yellow, and that if checkfail is present the scoring color should be red. If that is not correct, we need a specification for when green, yellow or red should be shown.

That looks good to me.

[wbnns]

Hello, just wanted to draft and confirm the initial set of features that users will be able to filter wallets by in the wallet selector for the newly redesigned wallet pages:

The proposed features that can be filtered are:

• multisig
• segwit
• bech32
• lightning
• hardware wallet support
• full node support
• coin shuffling

Any suggestions for revision - features to reword, add or remove?

[Cobra-Bitcoin]

That looks good to me. Don't really have any problems with it. Especially like that coin shuffling is there since we really ought to let wallets good for preserving Bitcoin's fungibility set themselves apart in some way.

[crwatkins]

I'm OK with that list. I think bech32 support should be full bech32 support (sending and receiving) whether we call it that or not. In addition, we may want to add legacy address support (p2pkh/p2sh) as not all of our wallets support both sending and receiving.

One addition item might be 2FA support. While this is implemented using multisig, I would propose that the existing multisig on the list be used to denote user designated co-signers, while 2FA support would denote a third party co-signing service based on some form of 2FA.

[wbnns]

@Cobra-Bitcoin @crwatkins Thanks.

[wbnns]

Just wanted to follow-up on the Privacy category that has been getting broken out in the mock-up:

Since avoiding address reuse is a requirement in order to be listed, not sure it makes sense to provide this on the frontend in the table for the purposes of comparison (all wallets would have a green circle). Also, since hardware wallets do not have privacy scores broken out, would recommend that we stick with the current logic/status quo, which is to show one privacy-related dot, with its color determined by the underlying subcategory scores. This will also help on tablet and mobile, where we have a much more finite amount of screen space to work with.

[wbnns]

We're drafting the feature reference table that will be used to originate what wallets will be displayed as features are toggled in the new selector.

Feel free to update the table if you know certain features are or aren't available for a specific wallet.

Cc: @natiwa, @Cobra-Bitcoin, @crwatkins, @maxwellmons, @alexcherman

[wbnns]

We're also now drafting the criteria descriptions that will be displayed on the criteria selection page as well as in tool tips when users mouseover the tooltip icons in the selector:

Edits welcome.

Cc: @natiwa, @Cobra-Bitcoin, @crwatkins, @maxwellmons, @alexcherman

[crwatkins]

@wbnns I think that's a great draft! Maybe we will figure out some tweaks as we use them in context, but I would be happy going with what you have.

By the way, I'm not thrilled with the term "Benchmarks", but I'm having a hard time coming up with anything better. Maybe we should call that list "Features" and then the second group (with 2FA in it) could be called "Capabilities"?

[wbnns]

@crwatkins Ok, great. What do you think about "Highlights" instead, since toggling one or more of the terms to follow would highlight criteria wallets rank well for?

[crwatkins]

I think "Highlights" might be even more confusing to me. What about a different approach of not trying to name them but describing what to do with them? Maybe something like "What is important to you?" or "Filter based on what is important to you".

[wbnns]

@crwatkins

We are going to do that in the wizard (not live yet), before the selector:

Maybe will just leave as "Criteria" for now, and can revisit later if we need to. The main reason it got changed to "Benchmarks" on this particular page was in case people skip the wizard we were thinking they might not understand what "Criteria" means. This might not be a concern, however, with the criteria descriptions mentioned above that will appear in the tool tips.

[wbnns]

@crwatkins

Hey, following up on:

I think bech32 support should be full bech32 support (sending and receiving) whether we call it that or not. In addition, we may want to add legacy address support (p2pkh/p2sh) as not all of our wallets support both sending and receiving.

While not all wallets offer full bech32 support. All wallets facilitate transactions to legacy addresses, no? If so, we were thinking it may not be necessary to include this item in the feature criteria, since toggling it would have no effect.

[crwatkins]

Wasabi and BLW do not support receiving to legacy addresses (as indicated in the spreadsheet).

[wbnns]

@crwatkins Ok, thanks.

[crwatkins]

By the way, I'm really happy to have the legacy address selector because we debated even listing wallets that did not fully support legacy addresses. Wasabi and BLW provide some advanced features that our community can certainly benefit from knowing about, but both have support issues when users attempt to fund the wallets, or receive payments, from sources that do not support bech32. This will go away in (hopefully short) time, but for the time being, it is nice to at least have a place to flag this.

[wbnns]

Just an FYI that we're drafting the feature descriptions in case anyone has any changes or feedback they would like to add.

Cc: @natiwa, @Cobra-Bitcoin, @crwatkins, @maxwellmons, @alexcherman

[wbnns]

Here is a draft of the OS Pros/Cons, the final section of new content as part of the wizard/selector, for all to review.

Cc: @natiwa, @Cobra-Bitcoin, @crwatkins, @maxwellmons, @alexcherman

[crwatkins]

Here is a draft of the OS Pros/Cons

I'm having a bit of a hard time coming up with pros and cons that are accurate, substantial, and seem to make sense. I fear this might be kind of like trying to come up with just a couple pros and cons for buying a car vs. buying a boat. You kind of know what you want already and the main points "Works on water/Doesn't work on land" are kind of silly while there are way too many finer points to choose from as being significant.

Are we married to pros and cons here? If we are, I hope someone else can polish these up because I'm having a hard time. If not, perhaps I could come up with a short statement about each instead.

[wbnns]

@crwatkins

The current list of pros/cons that we've put together seems like a good start.

I fear this might be kind of like trying to come up with just a couple pros and cons for buying a car vs. buying a boat. You kind of know what you want already and the main points "Works on water/Doesn't work on land" are kind of silly while there are way too many finer points to choose from as being significant.

We don't recommend removing this because we don't think we can assume that people, especially the massive number of new users, will already have an idea of what kind of wallet they want or that they would be able to easily weigh the pros/cons. For others, we present the option to skip the wizard.

[crwatkins]

I don't think we should remove the description; I just couldn't wordsmith the pros and cons into something that I found balanced enough (some qualities seem like they would equally apply to other platforms, but weren't listed).

We already have descriptions of these platforms embedded in our scoring text. So I started with those, and tried to cover all the important items in the pros and cons listed and wrote a one sentence description of each. How is this?

---

Mobile Mobile wallets are useful for in-person transactions and are generally safe from other apps and malware that you may download, but they may be easy to steal or lose making it important to understand PINs/passwords and backups.

Desktop Desktop wallets generally have richer, more advanced features and run on platforms with sufficient resources to provide full local validation of transactions in addition to being able to more easily import and export information, but are in general more susceptible to malware that may be on the platform.

Web Web wallets can be accessed from any web browser, but in general are considered to have more risks than mobile or desktop wallets and may require more trust in the wallet provider.

Hardware Hardware wallets are devices suited to storing larger amounts by providing additional security to software wallets by storing keys offline and requiring a connection to your computer to manage funds.

[wbnns]

@crwatkins

Ok, understood, thanks for clarifying and also for the additional text to compare. Here are mockups for each version --

Version w/ bullet points (pros/cons):

Version w/ description underneath: