politeiaww: Add user plugins.

[lukebp]

This issue will serve as the parent issue for adding user plugins to politeiawww. This work was funded by the Pi 2021 Q3 proposal.

Proposal Text

Politeia is split into two layers: politeiad and politeiawww.

politeiad is the data layer. It accepts and stores records. A record is an arbitrary set of files. A record can be a proposal, a CMS invoice, a forum post, or any other set of files.

politeiawww is the user layer. All user data is saved to a database in this layer. This includes data such as login credentials, user sessions, notification preferences, and any other type of data for user accounts.

The recent Politeia v1.0.0 release adds a composable plugin architecture to politeiad. A politeiad plugin extends a record with additional functionality like adding comments to a record or voting on a record using dcr tickets.

This same plugin architecture now needs to be added to the user layer in politeiawww. A user plugin will extend a user with additional functionality. Examples include adding a DCR paywall for specific user read/write actions or compiling github statistics for a user that is submitting a CMS invoice. This is going to be the largest chunk of work for this proposal since it includes refactoring the 25% of the codebase that the v1.0.0 release did not cover.

Listed below are the user layer deliverables:

• Add a composable plugin architecture to the user layer. This will require re-writing the user database implementation in a manner that supports composable user plugins.

• Add a user API that is designed for plugins. The user API will need to be re-written to accommodate the plugin model and the new database implementation. The existing user API will be deprecated.

• Add a DCR payments plugin. The proposal system currently has a feature that requires users pay a registration fee in DCR before they can submit data and a feature that requires they pay a fee in DCR to submit proposals. This functionality will be abstracted into a generic plugin that allows the server operator to configure one-time and recurring paywalls for server operator defined read/write operations, payable in DCR.

• Remove scaling impediments. There are a few memory caches in politeiawww for user data that are preventing politeiawww from being scaled out. Since we are re-architecting the user layer, we can use this opportunity to remove all impediments to scaling politeiawww and running multiple politeiawww instances.

• Make email optional. A site that wants to allow anonymity should not require the use of email. Since we are re-architecting the user layer, we can use this opportunity to make email optional. A user will have the option of authenticating using their identity (key pair that is specific to the user) and a TOTP.

Implementation

1. Add a base user data model and API routes.

• The base user will contain the fields and functionality that all users will share. This will be very limited.
• Additional user functionality will be layered on via plugins.
• The login method should be able to be replaced by a custom plugin login method.

2. Add the plugin model.

• Ability to extend a user with additional functionality via plugin commands.
• Ability to save data clear text or encrypted. Clear text data should be queryable via SQL.
• Ability to add user permissions onto the record routes and record plugin commands.

• Ability to hook into record and record plugin routes.

  3. Add user plugins.

  • auth - provides various user authentication methods.
  • identity - provides user key creation, storage, and signature validation.
  • ntfns - provides user notifications.
  • dcrpay - provides dcr payment functionality.

4. Add a user database CLI tool.

The new user layer will require a CLI tool for interacting with the user database. The tool will need to allow for plugin specific commands.

This is a good opportunity to start experimenting with golang plugins since the scope of a CLI tool is small, allowing us to experiment in a rapid, low risk manner.

Golang plugins

One of the long term goals for Politeia is to use a plugin architecture that allows the plugin code to be hosted outside of the Politeia repo. This will let third-party developers use Politeia as a general purpose platform that can be built on top of. Golang plugins may allow us to accomplish this. Hashicorp's go-plugin could also be a potential solution.

[lukebp]

UX pain point that should be fixed.

• User registers an account with email abc@example.com.
• They don't login for a while and forget that they have an account.
• They come back at some point and try to re-register the account abc@example.com.
• We don't show a email already exists error because we don't want an attacker to be able to ascertain what email addresses have politeia accounts. The request returns succesfully and no error is shown.

A better way to do this would be to send an email to the abc@example.com email address letting them know that someone attempted to register a new account with their email. If the person registering the account is the owner of abc@example.com, they now realize that they've already registered an account and can procede accordingly. If the person registering the account is an attacker looking for information, they still are not able to ascertain anything valuable.

[amass01]

Lately users are experiencing an UX issue with identity verification:

If an user creates a new identity and without verifying it the user creates another new identity, if the user tries to verify the second identity the verification fails. An admin needs to manually navigate to the user's account page and invalidate the first verification token, then the users can go and create a new third identity and verify it. We would like to improve this behavior as part on this issue effort.

[lukebp]

Admin actions must be logged and saved to the database.

[lukebp]

Create a key type for the user identity so that additional key types can be added in the future. Example, maybe we want to add the same key format that matrix uses so that the same key can be used for matrix and politeia.

[lukebp]

Matrix signup workflow with optional email.

Matrix login workflow.

[lukebp]

[WIP] auth plugin

Basic authentication

• Allow username with password authentication.
• Allow email with password authentication.
• Email will be saved encrypted in the database.
• Passwords are saved in the database as a Blowfish hash.
• Add a lookup table that maps the hash of the email to the UUID. This is required to allow for email with password logins.

Two-Factor Authentication (2FA)

The only type of 2FA that will be allowed is using a TOTP. By default, 2FA will be disabled.

Email Verification

Update Email

A user's email address can be updated or removed from their account.

Update Password

Reset Password

Future Roadmap

• Add enable 2FA to politeiagui user registration workflow.
• Add the ability for user's to reset their password using their user identity (public-private key pair) and a TOTP. This will allow user accounts that have not registered an email address to still eb ableto to reset their password.
• Allow passwordless authentication. The user would authenticate using their identity and a TOTP.

[lukebp]

Email will be optional. If you don't have an email set you need to have 2fa enabled. Updating things like your identity will need to require verification with whatever 2fa option you have set (email or TOTP). We need to make sure to add in rate limiting for anything that creates new data on the backend, like updating your identity.

[lukebp]

The user details page should use the user's username in the URL, not the user's uuid. Using the uuid is a bad UX.

Since we're using the username in the URL, we're going to need to make the make the allowed username character set more strict. Look at the character set that twitter and reddit use.

We need to be able to handle users who already have usernames with special characters in them. The URLs will be broken. What we can do to fix this is to strip any special characters from the username URL. Users will also have the ability to change their usernames. If they choose to update their username, they'll have to use the more restrictive character set.

[lukebp]

Add notifications for author proposal updates.

[xaur]

Making email optional will finally remove the involvement of email third parties (which can spy or block access) from Politeia<>user interaction. It will improve engagement of users who value this and help onboarding new ones.

One negative effect of removing the email is a loss of a notification channel with the user (who chooses to not use email). We will need an alternative channel and there are two high-level approaches: on-site and external notifications. For on-site we can borrow UX ideas from GitHub notifications or Reddit inbox views. For external we can look into DMing users on Matrix. The guiding principle I like is "help users not miss any stuff".

Rich notifications features will require a flexible design - the doc linked in #736 may help with ideas.

[xaur]

Re authentication, starting with TOTP only is pretty good.

Looking forward, I would also keep WebAuthn/FIDO2 (security tokens) in mind when designing abstractions.

A nice benefit of authenticating with a signature is it doesn't depend on time. Sometimes it is annoying to have to sync system clock to log in.

Allow passwordless authentication. The user would authenticate using their identity and a TOTP.

Auth idea seen in the wild: use a concatenation of a short PIN and a TOTP code instead of a password. This way you enter 10 digits instead of 6. I have not investigated why this was done, but there must have been a reason to add a PIN.

[xaur]

recurring paywalls for server operator defined read/write operations, payable in DCR.

Recurring paywalls would be a nice general-purpose feature for some deployments.

Not critical now, but a more coin-agnostic design would help Politeia adoption by other communities (which is desirable, IMO).

[xaur]

Admin actions must be logged and saved to the database.

Rewriting the user layer is also a good opportunity to think about data transparency.

First it is moderation, specifically user changes initiated by mods/admins. Things like bans should be publicly auditable (admin who did it, timestamp, reason, signature).

It is also a good time to plan for flexible restriction tools like user mute/ban/suspend (temporary or permanent, site-wide or local to some thread).

An existing system to learn from is Reddit. Over 16 years of operation they did not add an easy to use mod log, but at least there are APIs and hacks that allow 3rd party tools to expose it. Mod logs for r/decred are viewable on modlogs.fyi and rbtc.live (there have been other tools but they died).

In terms of outreach, this feature will help us attract people who value this level of transparency.

Second is transparency of user-initiated changes to their accounts. I think things like history of username or public key changes should be/stay public evidence.

[xaur]

rate limiting for anything that creates new data on the backend, like updating your identity

An extension of this and "one-time and recurring paywalls" is user quotas where you "pay per byte", optionally with some free daily bytes to use (non-accumulating). This will put a fun disincentive to writing long walls of text for people like me :D

I think we don't need this in short-mid term but it is the idea I keep getting when I see communities that lack funding to even sustain a forum.

[lukebp]

The identity package should standardize how the identity is saved when exporting to disk so that identities downloaded from the browser and CLI tools are compatible.