[Preliminary] Cross-device sync method and data format

[akinwale]

Problem Statement

In order to provide a unified user experience across multiple devices, some user data needs to be synchronized between the devices.

Requirements for Version 1

1. synced data is defined as the following info for each account in a user's wallet:
   • account_id
   • account type
   • account name
   • seed
   • private key
   • public key
   • address gap, recieving address max gap, receiving address max uses, change address max gap, change address max uses
   • channel private keys
2. synced data are kept synchronized between the user's apps (desktop, mobile, lbryweb)
3. synced data are accessible only to the user. LBRY cannot access the synced data in any way. this may require the user to store a secret value, without which they cannot access their synced data.
4. the definition of synced data may change in the future. our solution should anticipate this and remain flexible to changes
5. we will define a threat model which describes what data we store, how it may be accessed, what types of access we do and do not protect against, etc

Requirements for ideal version

1. if the user makes the active decision, they may enable "recovery" of the synced data to protect them in case they lose access to their secret value. this won't happen in the first version, but we need a clear understanding of how we will add this on in v2
2. ideally our solution for 5 would still keep the synced data inaccessible to LBRY, but if that is not possible, we will store the recovery data securely.

[BrannonKing]

Some of this data needs to be stored on a central server...

Let's cut this line: there is always another way in.

I would contend that media playback positions don't need to be sync'd across devices; I watch different things on my phone than my Roku.

[kauffj]

I think there are going to be a lot of questions about specifically what's synced and the format of it. Rather than trying to get every aspect of that correct right now, let's focus on what we need to be agreed on to allow @akinwale to move forward on solving the mobile wallet deletion problem. A first version of syncing user state can come in the next iteration.

With that goal in mind, let's boil down Akin's proposal to this:

• All state necessary to synchronize user experience will be packed and unpacked from a single JSON file.
• This state will be synchronized by a "central server". @akinwale omits more details on this - presumably internal API endpoints?
• Some state will be dedicated to additional channel certificates, media history, settings, and other things, but this precise design is omitted.
• version will be a top-level JSON value.
• data will be a top-level JSON value.
• The wallet seed will be stored as a top-level value inside of data.

{
   "version": 1,
   "data": {
       "seed": "who what where ... when"
    }
}

This seems sensible to me (it's simple :grin:), while allowing us to evolve it as we add more sync capacity. It probably makes sense to perform an additional encryption step on the seed itself so it's less easy to accidentally litter the seed somewhere (a response is cached, or accidentally written, etc.)

[kauffj]

@eukreign can you confirm wallet seed is sufficient to restore channel access?

[tzarebczan]

@kauffj it is not, but would be for new channels created after https://github.com/lbryio/lbry/issues/1433. So we'd still need to support legacy ones created before this.

[eukreign]

adding to what @tzarebczan said: while deterministically generated channels will allow them to be recovered this does not benefit channels that you imported from someone else... meaning that only the wallet which created a channel can deterministically recover that channel.

in order to recover the imported channels either we'll have to back them up on a central server - or - user will have to ask the person who gave them the channel to give it to them again.

[akinwale]

The proposal has been updated with details about encryption of the seed.

[kauffj]

Is there a simple/sensible way to include channel certificates along with the wallet seed if the seed is insufficient to restore them?

[tzarebczan]

Some questions/comments/concerns:

1) Will we be forcing users to enable encryption before syncing their wallet?

2) Will we allow users to sync more than one wallet account? Will we be merging accounts from existing wallets (i.e I have 2 installs with 2 wallets, what happens?)? Depending on the answer, we may need to change the way accounts are handled on the wallet or how we present multiple accounts in our apps.

3) We will need to handle the case where they decrypt/re-encrypt wallets

4) Do we want to offer 2FA as an additional layer of security?

5) Users should be able to opt out of syncing after initially agreeing to do so.

[nikooo777]

I have an additional proposal that we can use as base of discussion:

To solve https://github.com/lbryio/internal-issues/issues/226:

Rules of the game:

• We only sync encrypted seeds
• We encrypt each seed a second time on our servers before storing them in the database
• The data in our possession MUST NOT be accessible by us

Overview of the process:

• user starts the app
• user has a new wallet with a new seed (the account is created automatically on startup)
• user is prompted to go through the verification process (they need a verified account/email before they can do anything)

Case 1:

• this is the first encounter with LBRY that the user has
• the user is prompted with a suggestion to safely sync their wallet on the cloud (our servers) Steps taken by the user:
• A password is required to encrypt their wallet. A fatass reminder/warning must tell them that if they forget or lose the password they will lose their credits (*1)
• Their seed is sent to our servers through internal-apis (*2) over HTTPS!!!
• All channel certificates are to be encrypted with a seed-derived salt (perhaps use the wallet seed itself?)
• Channel certificates are sent to internal-apis just like the wallet seeds, additionally their public key or claimID is sent to identify their channels (*4) [might not be needed] Steps taken on our side:
• Internal APIs further encrypts the seed with a salt derived from their user (or anyway unique to the user) (*3)
• Interal APIs stores the double-encrypted seed in a table
• Internal APIs encrypts and stores the channel certificates in the same table Further steps from the user:
• Whenever a certificate is created or abandoned they should notify internal-apis so that they are added/deleted from the database

Case2:

• the user had already synced their wallet from a different device
• the user is prompted with the option to recover their wallet
• the user can have multiple wallets synced on the cloud, however to begin with, only one will be allowed
• the user choses to recover their wallet
• the seed is decrypted on our side (second layer encryption) and sent to the user over https
• the user is prompted for a password to decrypt their seed
• the SDK pops the default account and replaces it with the fetched account setting it as default account
• the (likely) empty account is not deleted, it will reside in the wallet as additional account (non default) (*5)
• upon successful restore of the account, the channel certificates are returned the same way
• the wallet seed is used to decrypt the channel certificates
• the certificates are imported in the wallet
• the app/sdk continues to monitor the APIs through an endpoint to see if new certificates are available
• the SDK monitors transactions to see if a certificate is abandoned and thus has to be deleted from the account

Others:

• The user could be prompted with the amount stored in their wallet prior to decrypting it if the public keys are available. This helps the user understand which wallet they desire when we will support mupltiple wallets

(1): We must compromise security vs ease of use, this password will be the weakest link of the chain and there is little we can do about it (2): Do we want a separate infrastructure/database for this? Would it make sense? Probably not. (3): The salt must becarefully considered as users can and will be merged, this seed must not change when 2 users are merged (4): Perhaps we don't want to know which user owns which channel (even though we could probably derive this information from the blockchain?) so we should consider encrypting the claimID or the public key client-side the same way as we encrypt the private keys before sending them. (*5): it's up to the SDK team and the app team to support multiple accounts on the app

Disclaimer:

• this is a first draft, it must be discussed, improved and built in smaller steps (TBD)

[tzarebczan]

I also had some ideas in this thread (https://lbryians.slack.com/archives/C1R07H02X/p1549039441425900?thread_ts=1549031389.407000&cid=C1R07H02X) on a mechanism that would allow us to support password recover - it entails creating a n of m cryptography / key sharing: https://en.wikipedia.org/wiki/Secret_sharing

Haven't worked out all the details on paper yet, but something along these lines should be possible and allow for secure account recovery via a 2fa/3rd party mechanism.

[skhameneh]

Rules / Notes / Assumptions

1) Wallet must be fully encrypted on the client's machine prior to usage.
This includes encrypting channel private keys. 2) The daemon must support generating private & public keys from encrypted seeds. (may not be needed, must verify flows for adding seeds) 3) First release will not support multiple wallets via API, data structures must be future proof (support it). 4) All wallet data records must be immutable., only account records can be changed.
Records can be edited by increasing the revision column where applicable. This is important to retain full channel integrity and prevent unintentional client data loss.
5) All web traffic must be encrypted with TLS 1.2 or higher.

See also https://observatory.mozilla.org/

Questions for Discussion

1) Do we want the channel TXO ID's (tx:nout) to be encrypted by the client? 2) Salting data with the salt stored on a separate machine is often considered best practice. This is not critical for the first release. We need to discuss this further today.

Data Storage

Accounts

A unique sync account. Accounts can have multiple wallets.

• id - Unique record identifier
• username - Custom username
• email - User email address
• password - Web account password, SHA-512 or better. Can be changed.

Wallets

Immutable wallet seed records, without revisions. Can only add, no editing/updating with first release. First release will not support multiple wallets, although no constrains prevent multiple from existing (for future use).

• id - Unique record identifier
• account_id - Corresponds to the id of the Account
• seed - Wallet seed. Must be encrypted by the user

Certificates

Immutable certificate records, with revisions. To update a record by matching txoid and account_id, the revision counter will be incremented.

• id - Unique record identifier
• txoid -
• account_id - Corresponds to the id of the Account
• revision - Integer, starts at 0. Increases every time the private key or active attribute is changed.
• active - Boolean
• private_key - Certificate's private key. Must be encrypted by the user

Data Reconciliation

1) Compare HASH(SORT(TXID:NOUT)) 2) If there is a difference, calculated what is added, dropped, and modified. 3) Add records, adjust old records as necessary with new revision's

[skhameneh]

@eukreign has suggested I rename Wallets->Account and Accounts->User. We can talk about naming conventions during the meeting.

[eukreign]

Other suggestions:

• on Certificate, drop active and revision
• on Account (currently Wallets), need the private_key in addition to the seed because imported certificates go into separate account which has no seed and just has the private_key.

[skhameneh]

• Users - Handled by existing email-based system, will need to explore adding mobile 2FA.
• Certificates - Will be encrypted by the user and each txoid/private_key pair will be a single value.
• • Can only be added, any ability to prune in either direction will be added later.

[skhameneh]

channel certificates should be encrypted #1888 https://github.com/lbryio/lbry/issues/1888 Wallet halt & sync daemon config for lbry-desktop (cross-device sync) #248 https://github.com/lbryio/internal-issues/issues/248

[eukreign]

Proposal No. 6

Terms & Definitions

Term	Definition
User	Stake holder in securing and accessing a set of accounts encrypted by a single password.
Wallet	A physical file on the file system which contains a collection of accounts. It is very much analogous to a real physical wallet carried by a person with fiat bills and debit cards.
Account	An account has a root public key at a minimum (but usually also a seed and a private key) and contains a collection of addresses to which and from which crypto currency payments can be made. An account is very much analogous to a debit card, a wad of cash or a checkbook found inside of a physical wallet. There are vanity accounts (no seed, only one address), hierarchical deterministic accounts (most common, usually has a seed and generates infinite addresses) and read-only accounts (no seed, no private key and has a public key only).
Client	An application which interacts with the user and negotiates data exchange between the sync server and lbrynet daemon.
Sync Server	An always-on publicly accessible server exposing APIs used by client to save wallet data.
account_id	A hash of the public key of the account, using the same algorithm used to create payment addresses from public keys in a hierarchical deterministic wallet (in the case of vanity accounts, the account_id is equal to the one and only address used for sending/receiving payments on that account).
password	A string of characters supplied by a user for the purpose of encrypting their data.

Overview

Sync process involves three components: lbrynet daemon, a client app and a sync server. Client app is responsible with deciding when and how the sync process will be performed and asking the user for input when necessary (such as to supply a password). The entire sync process can be broken down into the following general steps:

1. At regular intervals the client app will query daemon for a hash of the entire wallet (lbrynet sync hash). (In the long run lbrynet will expose a subscribable event stream of change events to avoid polling.)
2. Client will then send this hash to the sync server, if sync server determines that the hash is different from what it has then it will respond with a copy of the encrypted wallet; otherwise it will respond with 304 Not Modified HTTP status.
3. Continuing with the case where the hash doesn't match, the client app receives a copy of the wallet from sync server and asks lbrynet to merge the downloaded wallet into the existing local wallet.
4. After merging and reconciling changes, lbrynet will respond with a new hash and data result which the client app can compare to the starting hash, if new hash is different then data needs to be sent up to sync server.
5. Client app will send the updated wallet back to the sync server if local is more up-to-date.

Sync Hash

lbrynet

Client app will call the lbrynet sync hash command to get a hash of the entire wallet (after first encrypting the wallet with supplied password):

$ lbrynet sync hash
abc123

sync server

Next the client will request sync data from the sync server by calling the sync /get endpoint and passing:

• auth_token (to identify user)
• hash (as returned by sync hash lbrynet command)

There are 3 expected responses:

• 204 No Content - server has no record of a wallet for authenticated user, no body.
• 304 Not Modified - local hash matches server hash, therefore no sync is necessary, no body.
• 200 OK - local hash does not match server hash, server responds with latest wallet data in the body:

  {
  "hash": "[hash of the wallet data on sync server]",
  "data": "[encrypted wallet data from sync server]"
  }

Sync Apply

Client app will perform a sync apply after server responds with 204 or 200 HTTP codes:

In the case of 204 No Content, client will call sync apply without a data argument in order to generate the bootstrap wallet data to be sent to sync server.

In the case of 200 OK, the client app will apply wallet data received from sync server and if lbrynet responds with modified wallet data client will forward this to sync server.

1. lbrynet bootstrap

When sync server has no record of wallet for authenticated user, client would call the sync apply command without data argument:

lbrynet sync apply PASSWORD

{
  "hash": "[hash(json.dumps(wallet))]",
  "data": "[encrypt(json.dumps(wallet))]"
}

2. sync server bootstrap

Client would then send the response from sync apply to sync server using the /set end point and expect a 202 Accepted response.

3. lbrynet sync

When sync server returns wallet sync data the client will ask lbrynet to apply the data:

lbrynet sync apply PASSWORD --data=DATA

If sync data is applied and there is new data that needs to be sent back to the server then lbrynet will return a different hash value (different from what is on server):

{
  "hash": "[hash(json.dumps(wallet))]",
  "data": "[encrypt(json.dumps(wallet))]"
}

Client app will take the response from lbrynet above, add the old hash which was sent by the server and perform the server sync using /set API by sending it the message:

{
  "old_hash": "[hash from the sync server]",
  "hash": "[hash(json.dumps(wallet))]",
  "data": "[encrypt(json.dumps(wallet))]"
}

[skhameneh]

Copied from Slack: @eukreign we agreed to have API calls for sync'ing a single account at a time. It is clear you are proposing to move away from that. I have asked multiple times for you to address how we will be able to append certificates to the data stored on the server in an immutable architecture. Please address the flow of this, it is missing in your proposal.

[lyoshenka]

I believe this has been finalized. I'm closing this. Please reopen if there's more to discuss.