Having the Core sign the state of an Account

This idea has side effects related to "dead accounts", "true cold storage", "sharding accounts", and "ensuring what Horizon sent me was what the Core SCP actually agreed on" that I think are pretty useful.

A few of us noticed that the Horizon SDK clients have little way of ensuring a Horizon endpoint sent the actual balances of an account, versus whatever it felt like sending. This led to my considering a way for the "Core" to sign the current whole account state and have Horizon send that along with the rest of the account details (recipient can then validate the data wasn't tampered with by the man-in-the-middle).

The proposal is to add a "core signature" and "core signer" field to an account.

When an effect modifies something material on an account, like a "subentry", or most of the account's property fields, the proposing node also generates a "Sign Account" effect and computes the signature for the account's new, whole, state and supplies its own peer ID as the "core signer". This "Sign Account" effect would be the last step in a transaction to sign every account that was modified this txn (This could even be pushed out to being part of the "Ledger Close" instead if that's an option).

When the SDK does a "LoadAccount()" it can validate the "core signature" on the account to ensure the client received what the core agreed to via the SCP by checking it with the public key associated to the "core signer".

This assumes that the clients have a reliable method of obtaining an accurate list of public keys for the core signers (aka they didn't come from the cracked Horizon node that simply replaced these new fields too); or that the client retrieve the account's signature from multiple servers to compare several of them.

In my mind's eye the signature would include anything that counts as a "subentry" on the account. This likely means "account details" on Horizon would be increased to include the "Offers" list so that all the subentries on the account were returned with the account query and the signature properly checked.

So how does this signature help?

You can now do a "real" cold storage for an account. By leaving only the account signer/signature records in the database; you can delete the rest of the account from the database. When/If the account is ever "reloaded" you check that the data being reloaded matches the signature / signer data on record.
It validates to the receiving client, that the client received what the core has agreed to

But it has some interesting other uses too:

This means you can "remove" dead accounts. While they are "removed" they carry no reserve requirement for accounts that might be paying the reserve fee for the account.
The account can't be transacted with from or to; so while the account is "offline" it can't be changed in any way.
The account details cannot be looked up directly. You can always recompute them through transaction history, but the account status is simply "offline" or "unavailable" (you can confirm it exists).
This enables "sharding" of account processing across the core collective. Since it'd no longer be required for a database to hold all data for every account; all nodes can hold only the signatures and signers for every account. They can then request on demand the data for accounts they'd like to participate in. The algo for determining this is a separate discussion; here I'm saying that account sharding by the core is enabled if the account states are signed, and the signature agreed to by the SCP.

Thanks all! I appreciate the consideration. Mike

This "dehydrate" feature would be quite interesting indeed to prune the ledger. I don't know yet if this would be enough to shard the ledger in some interesting way (the main limitation right now is the order book). As for enabling light weight trustless clients, I think we'll have to do something like this at some point: the way I think about this is that we'll have to look at which data sets need to be verified like this and have simple structures (most likely Merkle trees) that point to this data. From what I can tell, it's fairly open ended right now.

Here are some of the questions that I think we probably want to be able to implement in a trustless environment:

account state (balance, but also trust line state, offers, etc)
ledger properties such as
- order book ordering (is offer X the best offer)
- compressed "histogram" order book view (allows to answer questions on price vs volume)

Reason I think we should wait before adding those is that I suspect the solution will look quite different depending on how we're going to do sharding (high transaction volume and large ledger size should have a minimal impact on how those trust less clients work); also there might be ways to encode rules instead of the actual full tree in the ledger, which brings me to the next point.

We have more and more features developed at the Horizon layer - and it seems to me that we need to solve this same problem at the Horizon layer as well as the complexity there will always be higher than at core's layer (there are more and more "aggregate views" consumed by trading clients) and the network needs to be decentralized at the horizon layer as well, not just at the core layer.

I agree with all those sentiments, especially pushing out the decentralization.

While it wasn't what I originally set out to create, by generalizing the idea a bit, it takes the whole architecture that way; the "Stellar Object Network" could be viewed as a decentralized hierarchy of signed structured data objects, transformed by ledger closes, using the SCP. The basic design rule is that for any level of the data hierarchy a node processes, the node must maintain all the ids, signatures, and signer data, for all siblings in that level; but can delete the data required to calculate those signatures.

A node validates the data it has is correct by validating the hash/signature on the parent container.

I believe it should be the case that if: a) there is consensus on the ledger close b) the ledger close dataset contains the hash of the "Account's Object Store" and c) using the ids, hash, signatures, and signers data of the accounts the node computes the same hash

then: a) the data it has on the accounts is correct b) the node can validate any account data it "hydrates" later on is accurate c) the same hierarchy validation technique can be repeated for other "Stores" d) the same hierarchy validation technique can be repeated for "subentry" account data

The biggest challenge I see is reliably distributing the pubKeys of the core; perhaps there's a way to link in the ledger close the list of the PubKeys of the validators that signed the account data? If the ledger close has a hash of that list, and a method to query that list, that could go a long way to helping the client "prove" the data it received hasn't been tampered with.

Focusing only on what data should be returned by Horizon's Account endpoint to make it possible for a client to validate the account is a really instructive exercise. A really good place to begin are validating the account's "subentries" (account main_data, balances, offers, custom_data, and signers). This also means that the account's current "offers", or at least the hash/signature/signer information of the latest offer's state, must be added to the Account results.

One glaring missing at the moment is a way to reliably communicate the true public key of the signer. I can only come up with having the Client check with and compare the answers from multiple core nodes.

Viewing the Accounts as containers of "signed structured data objects" and making the Accounts themselves "signed structured data objects"; and the "Accounts Object Store" that holds them a "signed structured data object"; the picture we paint is that Stellar is maintaining a hierarchy of "signed structured data objects" that are transformed, with their "current state integrity check" information baked in, by ledger close events.

The information in the ledger close then provides data integrity in two dimensions; the first dimension is "over time" which the ledger close already does by signing the "transformation links" happening with each ledger close (the ledger_hash). The second dimension, which I guess is "over space", is validating the final state of the post-transformed object hierarchy in its entirety for each ledger close.

If you took a snapshot of the data in the horizon database, and a single ledger entry; you could validate if the state of the data currently loaded in that snapshot was consistent with that ledger entry.

To enable "sharding" the participants must maintain a container's identifier, the integrity check data for the container, and the full set of top level object ids, hash, signatures, and signers (the integrity check data) for the objects in the container. From there, a node could dehydrate, and hydrate, integrity checked objects and know that it was in full sync with the rest of the network.

It does outsource some trust to the rest of network because that node no longer sees or validates the transformation of all the objects, only the parts it's watching (perhaps some work could be done in this area to ensure objects have sufficient distribution in enough quorum slices to be trustworthy).

Maybe a node can participate in multiple quorums, the one that's hand built to select the accepted transactions that we already have, and the other dynamically built based on the hydrated objects that handles the consensus on the final transformed state resulting from those transactions.

In essence, it feels like this is "Squaring the Data" in the ledger, we can already tell that each transaction set is valid; this then validates that those transactions were also applied correctly.

The main thrust of what I'm suggesting here is the generalized concept that if you transform a "hierarchy of data" into a "chain of hierarchy levels" where the hash on an entry comes from the list of ids, hash, signers, and signatures of the entry's contents; then you don't need to keep all the contents of all the objects.

Proving the signer of the entry is authorized, and that the signature is valid, is all that's required. The question then becomes "what key(s) signs?", and I suggest the key of the actor proposing the new state.

As I understand it, the "votes" are for a proposal by a particular node; if each node that validated a transaction offered a signed update to the data hierarchy for each level it modified, that should be enough to make the whole thing work. The rest of the network would converge on the proposal from a particular node; and that node's key becomes the "signer" for that ledger close.

Thanks! The next question is, are there any ways to integrate or test this to try it out / get started?

Hey @MikeFair — what do you feel you need here next? Do you want to move this forward with a CAP proposal (along with any additions that would probably need to be addressed at the Horizon/SEP layer?)

Hey @MikeFair — what do you feel you need here next? Do you want to move this forward with a CAP proposal (along with any additions that would probably need to be addressed at the Horizon/SEP layer?)

I would want to see/need a bit of a discussion/agreement on:

(1) how to calculate the "signature" and "signer" fields in a way that works amazingly well/easily with the consensus algorithm. This creates the ability to verify that the account hash came from a ledger state transition approved by the network, otherwise the whole thing falls apart (I think BLS signatures will do the trick, but am not aware of other available options or the risks to using them). Or said another way, this requires "the nodes of the network itself" to create a group signature that we can prove/trust.

(2) figuring out a trustworthy source for end user clients to get the ledger's public key to test account signatures with (and it can't be the same Horizon Node it is getting everything else from; and should probably be direct from some highly trusted core node(s).); this is kind of a chicken and egg situation and maybe is not perfectly resolvable. Horizon nodes can lie to your client about anything it wants and there is no easy way to tell it has done so. Having some way to reasonably fetch a pubkey(s) (like perhaps a DNS TXT entry?) to compare many data sources or a chain of trust (signing certs) that makes it easy to discover if a Horizon server (or any other type of server) sends fake data is required. This is one thing that should probably always come directly from the core nodes themselves.

(3) general consensus this is a worthwhile idea overall. I can't see any meaningful downside myself. It's a non-trivial change. The end result is the concept of "network signatures" where the nodes themselves are able to sign stuff collectively in a way that it takes the proper number of nodes in the network to sign something, no node gets to see the secret key of any other node, and as long as we can get the list of public keys (node ids) used in the ledger close, we can validate the signature. [Using the BLS scheme everything rolls up to a single key and signature for each ledger close, which I think we all would like.]

Using a combination of IPLD (from IPFS) and BLS (Bohen-Lynn-Shacham) signatures; each validating node "owning" that account (or account pair for trades) could fetch the hash for the account's new/updated "latest close state" (which comes directly from the account's new IPLD address) by adding the account directly into IPLD (this also could move the work of constructing the JSON responses into IPFS/IPLD and out of Horizon), and each of the quorum groups should be able to successfully combine their collective public keys to make a single distinct public key for each ledger close and the various signatures of that account's IPLD address (its MultiHash value) to get a "Ledger Signed Account State" that can easily be validated as having included a set of participating nodes in that ledger close event.

As I've been reading/learning about them, the way the BLS math works, it seems the order of signature or key combining doesn't matter as long as the total set of keys and signatures ends up the same. This seems like it could be very compatible with the p2p messaging and ledger close process of the SCP. In other words, as each node combines its signature and public key to the proposed ledger, the order of combining the keys/signatures doesn't matter (it simply must prevent itself from signing the same proposal twice). As long as the final set of included nodes is the same, each peer can calculate and validate the "ledger close public key and signature for the ledger close state hash". As nodes come and go, the resulting ledger public key will change, which is exactly what I think we'd want.

What I love about this idea is that every ledger close has a single public key (which can be easily looked up), the signature on the objects modified (accounts, order book, etc) during that ledger close can be tested using that public key, and no one can get the secret key of that ledger close without getting the secret keys of all/enough of the participating nodes.

When a validator receives an operation to reload a dehydrated account, if it can get the last ledger close id and the account signature; it can then get the public key for that ledger close and can validate the hash on the data being uploaded.

So this basically means each node only needs to keep track of three pieces of data per "object": 1) Object ID -- so we can know what object/entity we are talking about 2) Last updated Ledger Close ID -- so we can easily fetch the set of nodes and get/create the pubkey 3) Signature -- so we can validate the object's details that are being restored

This concept creates an information DAG where at the very top level is a ledger close, and from there it can branch into several "stores" of hierarchical information.

at the top is the ledger store which has the pubkey, a state hash signature, and the ledger close state hash (which is a combination of its children store state hashes)

store 1) transactions processed this ledger close; state hash signature/state hash transactions

store 2) new account states as a consequence of this ledger close; state hash signature/state hash full account state data signed by the network itself

store 3) new order book state as a consequence of this ledger close; state hash signature/state hash full order book state signed by the network itself

Each of those stores then have their own children, and so on and so forth.

I included the order book as a separate store to separate the need for nodes to keep track of both the full set of accounts and the order book globally. The two things can be tracked as two state independent objects and different nodes can be participating in them.

@theaeolianmachine I hope that clarifies where this proposal is atm.

Adding some mechanism for the network to "sign something" is not currently a concept. Is anyone either for or against the idea of using BLS signatures to create a singular "Network Public Key" that can sign object states after they are transformed by the network?

@theaeolianmachine

My apologies, I originally proposed a single core node acting as the signer for an account state, and in this last comment I switched to proposing BLS signatures instead. The key reason for exploring BLS signatures is finding a sane way to create a trustworthy signing key that works with the SCP.

It was in researching the idea of "A group of computers signing something" that I found BLS. So the discussion in my "point 1" is exactly this; what signature do we trust here?

I didn't know about BLS when I first proposed the idea; so trusting a single node, where its work was validated multiple times by other nodes and then accepted, seemed like the only reasonable option. The challenge was proving that the signing node id was in fact the signer accepted by the network in that ledger close (it's easy for an Horizon to put whatever public key it wants as the signer field). Using a composite key like BLS seemed a better way to address that shortcoming/requirement (at the expense of a little more work during validation)...

@MikeFair could you post your last few comments to our mailing list, and link to this issue? You can join it here. I think you'll likely get more traction towards creating a draft there. Happy to help here as well.

stellar / stellar-protocol

Having the Core sign the state of an Account #81