Add a third data item on Managed Data for data sync and encrypted values

[MikeFair]

I've been working with account data lately which is currently a straight kvp.

It would help tremendously if there was a third field as part of this record to host a "field revision id". It would aid in syncing with outside third party apps and as part of a field encryption key.

Each time the data field was written to, this value would change. This value is part "field sequence number" and part "data hash".

By comparing this "revision id" value to a value the external systems have stored locally for the field, these external applications would be able to determine (a) if they have the most recent value for a record, or (b) if the data value needs updating from their own local values. Just comparing to see if the two values are different doesn't help in determining which of the two values is the more current value.

Also, because the value changes every update, it can be part of preventing reusing the same encryption key twice by incorporating it as part of the symmetric key value that encrypts the field data.

Thanks, Mike

[vogel]

Do you mean DataEntry values from Stellar-ledger-entries.x (added with ManageData?). These are held by LedgerEntry structure that contains lastModifiedLedgerSeq number, and this looks like the "field revision id" you need.

Not sure if horizon exposes that field.

[MikeFair]

Yes that's the Manged Data I'm talking about. :)

For data sync purposes, that one works great.

For encryption purposes (the use case I'm coding for first), the writer can't predict what that value is going to be before it actually happens (so can't use it as part of the encryption key). Readers can't really hunt for the right sequence number "near that one" because they don't know what the values are supposed to be; so I'm not sure how'd they test to see if they got it right (with a data hash, they could check the value to ensure it decrypted properly).

And no, Horizon doesn't currently expose it; and while I didn't personally look, my understanding is it also isn't in the postgresql database schema at the moment either.

[jedmccaleb]

Isn't the one in the ledger always the most current?

[MikeFair]

No, not if the value on the ledger is a publication of a value sync'd through third party systems.

Picture two federated environments that are sync'ing a value for the same userid. EnvA and EnvB (say two different data centers) both have the ability to update or be updated by the data on AcctC.

Neither needs "exclusive control" over the value, but they want to ensure their internal systems either reflect the most recent value, or push the updated value to the ledger.

It's easy to detect that the value EnvA locally has is different than the value on AcctC, but it can't immediately tell which "direction" the update should go. This third 'revision id' value can rectify that.

LedgerSequenceNumber is almost enough, but it's not predictable in advance. The reason that matters is that the "writer" needs to be able to communicate/store what it wrote.

---

Without thinking about encrypted data values, ledger sequence number might be enough. There's probably a way for a third party database writer/sync algo to figure out the sequence id of its update.

For encrypted values though, this third field could also serve as a "salt" or "initialization vector" for an encryption key to ensure that the same encryption key is never used more than once.

You'd take the "base secret key", modify it by this third field, and that new derivative key would be the decryption key for the field's value. You could require that an encrypted value must use part of its data to store this secret transform, but I believe a third writable field would be "better"...

[theaeolianmachine]

Hey @MikeFair — if you have any interest in pushing this forward, writing a CAP draft, or moving conversation to our mailing list (link in the main README) to get some feedback will push things along further.