Declarative format for service specification

[oudod]

Rendered version

[alekseysidorov]

I think that field sizes declarations is redundant. So what do you think about nested types? And what about documentation generation for these format?

Also, can you add link to the rendered version of these rfc to description topic?

[vldm]

Rendered

Can you explain, why using TOML?

As for me TOML isn't a perfect choice, because it has a lot of separating punctuation characters, thats make it annoying. Additionally toml is a flat, but we have some kind of hierarchy. What about "field_type", does this format specific for Rust only? What format we should use in describing fields type?

If we use rust, may be some Rust structure composition, is a better choice?

[slowli]

0. Have you considered WSDL / OpenAPI (fka Swagger) spec? If yes, please state clearly why they are inappropriate for our case. Also please take a look at other IDLs (e.g., the OMG one); they may have useful concepts (such as namespaces).

1. As @vldm has said, the description format looks dangerously Rust-specific (e.g., I don't quite get why the type of transactions.TxTransfer.from is &PublicKey, not PublicKey; what does a reference signify?). The field_size fields seem redundant, unless I'm missing something.

2. As for a specific serialization format, imo, JSON/YAML/TOML are not very different; they use the same underlying data model. As a personal preference, I like YAML a bit more. So, something like

   types:
   TxTransferRequest:
   type: struct
   doc: |
      `TxTranferRequest` is used in `TxTransfer` transactions
   fields:
   - name: from
     type: PublicKey
     doc: Originator of the transaction
   - name: to
     type: PublicKey
     doc: Recipient of funds
   - name: amount
     type: u64
     doc: The amount of funds transferred in the transaction
   transactions:
   TxTransfer:
    messageId: 128
    transport:
    - type: http+json
      method: post
      endpoint: "{{basePath}}/v1/transfer"
    requestType: TxTransferRequest
    doc: Transfers coins from one account to another

   (Note that YAML/JSON/... could be extracted from a more "natural" C-like IDL descriptions, like the ones used in CORBA; we only would have a syntax for this language and use a parser.)

Notice a couple of things here:

• doc fields for documentation (i.e., what @alekseysidorov seemed to ask with generation of documentation)
• TxTransferRequest has type struct. Besides structs, we have var-length arrays, and aliases (e.g., PublicKey is an alias for a binary buffer with fixed length 32; please design IDL in such a way that all these can be expressed within the language (i.e., PublicKey together with other standard types is defined somewhere in the "standard library").
• The fields within the struct are ordered
• transport in TxTransfer provides (so far, a bit hypothetical) flexibility regarding transport for endpoints (besides POSTing JSON serialization of transactions, transactions could theoretically be transmitted via WebSockets, SOAP or other means)

[slowli]

Some thoughts here:

YAML/JSON/TOML aren't IDLs, they are ways to represent information about the service. If we really want to, we could first define the JSON/... scheme and then define a BNF for a custom IDL and use an existing LL/LR parser (e.g., ANTLR) to convert from this IDL to JSON-structured spec. Can probably wait, though.

If we want to make the service description serializable (which could be useful for authentication and storing the service description in the blockchain state, a-la information_schema), it probably needs to conform to the serialization format. This is a problem because afaik we do not have a specification for the format yet. Hypothetically, the specification could be a struct, types, transactions within it could be maps, etc. There are several interesting questions with this approach:

• How to reference types? Types could be content-addressable (i.e., addressed w/ a hash of the type declaration; cf. structural typing) or declared sequentially within each service and the core (i.e., types would be an append-only list; cf. nominal typing). In the first case, the reference would simply be the type hash; in the second, an enum might be required (e.g., a variant for addressing core types, types within the same service, and types from other services).
• How to identify transactions / read requests? Is message_id enough?
• How to serialize documentation? Is a separate map required for it, or should inline documentation do the job? (The first option seems more extensible, e.g., for localization.)

I don't quite understand how storage is specified. Logically, the storage should be a map table_id -> table_type, but it would not take hierarchy into account (e.g., the wallet history).

I don't quite understand how read requests are specified, particularly, their responses.

Maybe, manual specification fo transports in the service description is a bad idea; there should be a standard way to may endpoints to a particular transport.

I'm thinking about prototyping the service description from the client side (see exonum/exonum-client#8 and this branch - both most probably need heavy reworking); doesn't seem like a particularly difficult task if one doesn't care about performance. The goal of prototyping would be to feed the service spec to the proverbial Service constructor and get a client-side service stub that would be able to post transactions / read requests and verify results.

[slowli]

Here is the repo with a JSON-based data schema declarations in JS (i.e., primarily oriented for the lightweight client) I'm working on now - slowli/exonum-types. Work in progress, obviously. This could form the base for parsing service definitions by the client and creating corresponding client stubs.