Restructure client code into a bundle of microservices

Problem

We're now (or about to be) facing a similar situation on the client side (pcli) as we were in pd prior to #488: we started with code designed to be an MVP, it's been extended beyond that scope, and we're now at the point where further extension risks blowing up the complexity of the code.

At the same time, we're getting a better sense of our needs for the client code, and there are more use-cases to support:

pcli itself: a standalone command-line utility for interacting with the chain;
galileo, the discord faucet bot (as a test-case for "an online service that transacts on-chain");
IBC relayers, which need to be able to at least create transactions, and possibly manage shielded funds to be able to pay fees;
active LPs, who would want to be able to manage liquidity (this is a speculative use-case)

The current client code (pcli + penumbra-wallet) was designed with a much more vague version of the above use cases in mind: penumbra-wallet was supposed to have reusable code that would handle Penumbra-specific logic (like how to scan a block, and what state to serialize as part of the wallet), which pcli would call to do whatever operations it needed. Other applications could use the penumbra-wallet code directly. However, then they have to use the Rust crate (trivial for Rust applications, potentially nontrivial for other languages), and implement the logic required to drive the wallet code, and this is a bunch of effort.

Solution

Instead, it would be useful to be able to completely offload responsibility for synchronizing the chain state and performing transactions to a standalone view service. This service would:

maintain user data in a database (SQLite seems like a good choice);
synchronize with the chain state on launch;
stay in sync with the chain state as long as it's running;
implement a gRPC service that:
- buffers all requests until the service is ready (synchronized with the chain state);
- provides queries for private chain state (UTXOs, balances, transactions, ...);

This service should be written as a library, so that it can be used in multiple ways:

by a standalone pviewd server that exposes the gRPC service (access control for the gRPC service is left as a deployment concern);
by a Rust application like pcli, which would create an instance of the view service internally, but call its (gRPC-derived) methods directly, then shut down once it was finished.

To clarify terminology:

The client protocol is spoken between a full node (the server) and a client wishing to learn about the public state (including encrypted data). This is split into two sub-protocols:
- The oblivious client protocol has requests that are intended to not reveal specific user information (e.g., downloading a range of CompactBlocks, or downloading information on every validator);
- The specific client protocol has requests that do reveal specific information (e.g., downloading a specific transaction the user is interested in, downloading information on a specific validator, etc);
The view protocol is spoken between a view service (the server) and a client wishing to learn about private chain state resulting from scanning;
The custody protocol is spoken between a custody service (the server) holding spend authorization keys, and a client wishing to have a transaction plan authorized. This custody service could be a "SoftHSM", or a threshold signing cluster, or an HSM, or a hardware wallet, etc.

Plan

Our current implementation model looks like the left, with all of the private user data bundled into a big ball in pcli, and all of the public chain data in pd. Instead, I think we should move to the diagram on the right, which separates viewing capability and spending capability:

          ╭   ┌───────┐                
  spending│   │custody│                
capability│   │service│                
          ╰   └───────┘                
               ▲     │                 
               │tx   │auth             
               │plan │data             
               │     ▼                 
          ╭   ┌───────┐                
   viewing│   │wallet │ tx submission  
capability│   │logic  │────────┐       
          │   └───────┘        │       
          │    ▲               │       
          │    │view private state     
          │    │               │       
          │    │               │       
          │   ┌───────┐        │       
          │   │view   │        │       
          │   │service│        │       
          ╰   └───────┘        │       
               ▲               │       
               │sync private state     
               │               │       
          ╭ ┌──┼───────────────┼──────┐
    public│ │  │     Penumbra Fullnode│
     chain│ │  │               │      │
      data│ │  │               ▼      │
          │ │ ┌──┐ app   ┌──────────┐ │
          │ │ │pd│◀─────▶│tendermint│ │
          │ │ └──┘ sync  └──────────┘ │
          │ │               ▲         │
          ╰ └───────────────┼─────────┘
                         .──│.         
                       ,'   │ `.       
                  .───;     │consensus 
                 ;          │sync      
               .─┤          │   ├──.   
             ,'             │       `. 
            ;   Penumbra    │         :
            :   Network  ◀──┘         ;
             ╲                       ╱ 
              `.     `.     `.     ,'  
                `───'  `───'  `───'

In this model, we split the existing client code into three entities:

a view service, responsible for synchronizing with the chain and providing views of private chain state;
the wallet logic, responsible for the "business logic" of preparing the transaction, doing proving, etc;
a custody service, responsible for holding spend authorization keys and signing transactions.

One aspect of this design is where the responsibility for building, signing, and submitting transactions should lie. If this were placed in the view service (in other words, identifying the view service with "the wallet"), we'd need it to know about how to sign transactions (or how to request their signing), which is awkward because it's mixing a weaker capability (viewing) with a stronger capability (signing).

On the other hand, if we define the view service's sole responsibility as providing a read-only view of the current state of some private user data, its role is neatly compartmentalized, and we have a clear progression from "more online + less capability" (with pd at one extreme) to "less online + more capability" (with the custody service at the other).

Moreover, we can arrange these roles into different components in multiple ways, for different use cases (note: horizontal scrolling):

 pcli (basic)                 pcli + pwalletd              pcli + pwalletd              web wallet + keplr           web wallet + keplr        
                                                                + hardware wallet                                               + pwalletd     
┌─────────────────────────┐  ┌─────────────────────────┐  ┌─────────────────────────┐  ┌─────────────────────────┐  ┌─────────────────────────┐
│ ┌───────┐           pcli│  │ ┌───────┐           pcli│  │ ┌───────┐       hardware│  │ ┌───────┐          keplr│  │ ┌───────┐          keplr│
│ │custody│               │  │ │custody│               │  │ │custody│         wallet│  │ │custody│               │  │ │custody│               │
│ │service│               │  │ │service│               │  │ │service│               │  │ │service│               │  │ │service│               │
│ └───────┘               │  │ └───────┘               │  │ └───────┘               │  │ └───────┘               │  │ └───────┘               │
│  ▲     │                │  │  ▲     │                │  │  ▲     │                │  │  ▲     │                │  │  ▲     │                │
│  │     │                │  │  │     │                │  └──┼─────┼────────────────┘  └──┼─────┼────────────────┘  └──┼─────┼────────────────┘
│  │     │                │  │  │     │                │  ┌──┼─────┼────────────────┐  ┌──┼─────┼────────────────┐  ┌──┼─────┼────────────────┐
│  │     ▼                │  │  │     ▼                │  │  │     ▼            pcli│  │  │     ▼      web wallet│  │  │     ▼      web wallet│
│ ┌───────┐               │  │ ┌───────┐               │  │ ┌───────┐               │  │ ┌───────┐               │  │ ┌───────┐               │
│ │wallet │               │  │ │wallet │               │  │ │wallet │               │  │ │wallet │               │  │ │wallet │               │
│ │logic  │────────┐      │  │ │logic  │────────┐      │  │ │logic  │────────┐      │  │ │logic  │────────┐      │  │ │logic  │────────┐      │
│ └───────┘        │      │  │ └───────┘        │      │  │ └───────┘        │      │  │ └───────┘        │      │  │ └───────┘        │      │
│  ▲               │      │  │  ▲               │      │  │  ▲               │      │  │  ▲               │      │  │  ▲               │      │
│  │               │      │  └──┼───────────────┼──────┘  └──┼───────────────┼──────┘  │  │               │      │  └──┼───────────────┼──────┘
│  │               │      │  ┌──┼───────────────┼──────┐  ┌──┼───────────────┼──────┐  │  │               │      │  ┌──┼───────────────┼──────┐
│  │               │      │  │  │              pwalletd│  │  │              pwalletd│  │  │               │      │  │  │              pwalletd│
│ ┌───────┐        │      │  │ ┌───────┐        │      │  │ ┌───────┐        │      │  │ ┌───────┐        │      │  │ ┌───────┐        │      │
│ │view   │        │      │  │ │view   │        │      │  │ │view   │        │      │  │ │view   │        │      │  │ │view   │        │      │
│ │service│        │      │  │ │service│        │      │  │ │service│        │      │  │ │service│        │      │  │ │service│        │      │
│ └───────┘        │      │  │ └───────┘        │      │  │ └───────┘        │      │  │ └───────┘        │      │  │ └───────┘        │      │
│  ▲               │      │  │  ▲               │      │  │  ▲               │      │  │  ▲               │      │  │  ▲               │      │
└──┼───────────────┼──────┘  └──┼───────────────┼──────┘  └──┼───────────────┼──────┘  └──┼───────────────┼──────┘  └──┼───────────────┼──────┘
   │               │            │               │            │               │            │               │            │               │       
┌──┼───────────────┼──────┐  ┌──┼───────────────┼──────┐  ┌──┼───────────────┼──────┐  ┌──┼───────────────┼──────┐  ┌──┼───────────────┼──────┐
│  │     Penumbra Fullnode│  │  │     Penumbra Fullnode│  │  │     Penumbra Fullnode│  │  │     Penumbra Fullnode│  │  │     Penumbra Fullnode│
│  │               │      │  │  │               │      │  │  │               │      │  │  │               │      │  │  │               │      │
│  │               ▼      │  │  │               ▼      │  │  │               ▼      │  │  │               ▼      │  │  │               ▼      │
│ ┌──┐ app   ┌──────────┐ │  │ ┌──┐ app   ┌──────────┐ │  │ ┌──┐ app   ┌──────────┐ │  │ ┌──┐ app   ┌──────────┐ │  │ ┌──┐ app   ┌──────────┐ │
│ │pd│◀─────▶│tendermint│ │  │ │pd│◀─────▶│tendermint│ │  │ │pd│◀─────▶│tendermint│ │  │ │pd│◀─────▶│tendermint│ │  │ │pd│◀─────▶│tendermint│ │
│ └──┘ sync  └──────────┘ │  │ └──┘ sync  └──────────┘ │  │ └──┘ sync  └──────────┘ │  │ └──┘ sync  └──────────┘ │  │ └──┘ sync  └──────────┘ │
│               ▲         │  │               ▲         │  │               ▲         │  │               ▲         │  │               ▲         │
└───────────────┼─────────┘  └───────────────┼─────────┘  └───────────────┼─────────┘  └───────────────┼─────────┘  └───────────────┼─────────┘
             .──│.                        .──│.                        .──│.                        .──│.                        .──│.         
           ,'   │ `.                    ,'   │ `.                    ,'   │ `.                    ,'   │ `.                    ,'   │ `.       
      .───;     │consensus         .───;     │consensus         .───;     │consensus         .───;     │consensus         .───;     │consensus 
     ;          │sync             ;          │sync             ;          │sync             ;          │sync             ;          │sync      
   .─┤          │   ├──.        .─┤          │   ├──.        .─┤          │   ├──.        .─┤          │   ├──.        .─┤          │   ├──.   
 ,'             │       `.    ,'             │       `.    ,'             │       `.    ,'             │       `.    ,'             │       `. 
;   Penumbra    │         :  ;   Penumbra    │         :  ;   Penumbra    │         :  ;   Penumbra    │         :  ;   Penumbra    │         :
:   Network  ◀──┘         ;  :   Network  ◀──┘         ;  :   Network  ◀──┘         ;  :   Network  ◀──┘         ;  :   Network  ◀──┘         ;
 ╲                       ╱    ╲                       ╱    ╲                       ╱    ╲                       ╱    ╲                       ╱ 
  `.     `.     `.     ,'      `.     `.     `.     ,'      `.     `.     `.     ,'      `.     `.     `.     ,'      `.     `.     `.     ,'  
    `───'  `───'  `───'          `───'  `───'  `───'          `───'  `───'  `───'          `───'  `───'  `───'          `───'  `───'  `───'

In particular, if pviewd is only responsible for presenting a view of the private user data, we could conceivably support its use as an information backend for a web wallet (e.g., a power user deploys a personal pwalletd on a VPS or a raspberry pi, or someone deploys a public pviewd in SGX, or ...), and if it's not responsible for transaction signing, we can use it contexts (like a web wallet) where we have an external transaction signer.

One thing that this model can't do, by design, is the current implementation of "pending" notes. Right now, in pcli, we modify the local state and implement an ad-hoc state change + rollback system to track transactions that we've submitted to the chain but which have not yet been recieved. This causes a huge increase in the complexity of the state management code, but the block interval is <10s, we're paying that increase in complexity for a very short time window, and we could alternatively just wait for the transaction to be confirmed before reporting success, in which case the problem goes away completely. Beyond the simplification of the state management code, not doing this also means that we only need to feed data through the view service in one direction (away from the public chain, towards higher cryptographic capability) rather than two.

As with the pd restructuring in #488, I think it would be better to do a parallel rewrite using our existing code as a reference, rather than a refactoring of the existing code. The big advantage of this approach is that we unlink the progress on this project from our other projects, we can merge in-progress code and compare it with the existing implementation, and we can avoid carrying over architectural decisions that might not still make sense. To do this, I'd suggest that we make new crates: view, custody, wallet-next and then pcli-next, which should eventually replace the existing wallet and pcli crates. The view crate would have the implementation of the view server, as well as a pviewd binary that stands up the server. The pcli-next crate would use the view server as described above.

[x] #610
[x] #611
[x] #614
[x] #612
[x] #613
[x] #615
- [x] #750
- [x] #751
- [x] #752
- [x] #753
- [x] #754
- [x] #755
- [x] #756
- [x] #757
- [x] #758

This work should be done before we substantially expand the scope of the wallet functionality, so starting now seems good.

penumbra-zone / penumbra