Declarative format for service specification

[oudod]

The issue was addressed in the roadmap.

Main difficulties:

1) Build Merkle proofs architecture by the data schema in declarative format.

Propose to use few levels of proofs (TBD detalize processes of forming and checking proofs):

• block
• wallet (or user ID)
• transaction

Related question: whether the root hash of a table can be used in a few places ?

2) Updating the root hash of the changed table in all related tables.

3) Language for the declarative format:

• JSON. Light clients use JSON anyway. So chosen format must be JSON-compatible.
• TOML. Seems to be more human-readable than JSON. The question is if TOML can be unambiguously converted to JSON and vice versa?

[oudod]

As stated in toml crate documentation TOML and JSON are interchangeable.

Also HJSON may be considered as human-readable language for the declarative format.

[slowli]

I don't think the format is the main issue; e.g., the Swagger spec supports both JSON and YAML, and in principle all these formats reflect approximately the same data schema. A better question is what needs to be specified; besides the proof format, I would think about specifying the format for events (which are now implemented via ~horrible hacks~ polling) and transactions (which presently are specified through execution, which I don't see as a long-term solution; design by contract looks better).

As for Merkle proofs, there are 2 different issues: specifying how proofs are structured (which seems easy) and specifying how the proof needs to be verified (much harder; I can only think of some declarative format a-la OpenTimestamps, perhaps more high-level, or some stack machine).

For example, suppose we have a cryptocurrency service, in which the data has multi-level hierarchy: there is a map of public keys to wallet information - user name and the root hash of the Merkle list of these transactions. Correspondingly, we have the following types (I use Flow type notation):

type WalletInfo = {
  username: string,
  transactions: ListView<Transaction>
};

type WalletView = MapView<PublicKey, WalletInfo>;

const serviceId = 1;
const walletsId = 1;
type WalletResponse = BlockWithProof<serviceId, walletsId, WalletView>;

type WalletRequest = {
  publicKey: PublicKey,
  fromTxIndex: number
};

where:

• ListView<E> is the Merkle proof for elements of type E
• MapView<K, V> is the Merkle Patricia proof for keys of type K and values of type V
• BlockWithProof<sId, tId, ListView | MapView> is the part of the proof corresponding to the block header.

That is, integrity checks would be contained within these datatypes deserialization (e.g., that the response JSON has the correct form, the hashes of Merkle trees match up, block header is authenticated, etc.). The external checks could be internalized given a pair [WalletRequest, WalletResponse] (i.e., check that the key in the returned MapView is publicKey from the request, and the indexes in the returned ListView start from fromTxIndex), but the problem is how to describe this process in the general case.

Hope this is somewhat helpful.