search

Concordium / concordium-rosetta

A server implementing the Rosetta API for the Concordium blockchain.
Mozilla Public License 2.0
5 stars 1 forks source link

readme

Concordium Rosetta

A server implementing the Rosetta API for the Concordium blockchain.

The application serves plain unencrypted HTTP requests. Any TLS connections must be terminated by a reverse proxy before the requests hit the server.

The server performs all on-chain activity against a node through its gRPC interface.

The project is written in Rust (minimum supported toolchain version is listed below). A great way to install the toolchain is via rustup.

Versions

Build and run

Prerequisites

The repository uses nested git submodules. Make sure that all submodules are checked out correctly using

git submodule update --init --recursive

IMPORTANT: This must be done after the initial clone as well as after switching branch.

Build

The command cargo build --release will place an optimized binary in ./target/release/concordium-rosetta. The application accepts the following parameters:

Docker

Build

IMPORTANT: Before building, make sure that submodules are checked out correctly as described above.

docker build \
  --build-arg=build_image=rust:1.73-slim-buster \
  --build-arg=base_image=debian:buster-slim \
  --tag=concordium-rosetta \
  --pull \
  .

Run

Exposing port:

docker run --rm -p 8080:8080 concordium-rosetta <args...>

Host network:

docker run --rm --network=host concordium-rosetta <args...>

See also docker-compose.yaml for an easy way of building and/or deploying an instance with sensible defaults using Docker Compose.

Rosetta

Rosetta is a specification of an HTTP-based API designed by Coinbase to provide a common layer of abstraction for interacting with any blockchain.

The Rosetta API is divided into three categories:

There are also mentions of a Call API for network-specific RPC, but it doesn't appear to be a first class member of the spec.

To learn more about the intended behavior and usage of the endpoints, see the official documentation and the example section below.

Implementation status

All required features of the Rosetta specification are implemented: Everything that isn't implemented is marked as optional in the spec/docs.

The sections below outline the status for the individual endpoints along with details relevant to integrating Rosetta clients.

Data API

All applicable endpoints except for the optional mempool ones are supported:

Construction API

All applicable endpoints are supported to construct and submit transfer transactions with or without a hex-encoded memo.

Indexers

Not implemented.

Call API

Not implemented.

Identifiers

Rosetta uses a common set of identifiers across all endpoints. This implementation imposes the following restrictions on these identifiers:

Identifier strings are generally expected in standard formats (i.e. hex for hashes, Base58Check for account addresses etc.). No prefixes such as "0x" may be added.

Operations

Rosetta represents transactions as a list of operations, each of which usually indicate that the balance of some account has changed for some reason.

Transactions with memo are represented as the same operation types as the ones without memo. The memo is simply included as metadata if the transaction contains one. Transaction types are otherwise represented by operation types named after the transaction type.

For consistency, operation type names are styled with snake_case.

The Construction API only supports operations of type transfer.

Errors

All success responses are returned with an HTTP 200 message. Errors are returned with an appropriate 4xx code if they're the result of the client input. Errors propagated from the SDK are given a 5xx code.

Examples

Construction API

Transfer 1000 µCCD along with a memo from testnet accounts 3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi to 4Gaw3Y44fyGzaNbG69eZyr1Q5fByMvSuQ5pKRW7xRmDzajKtMS.

The command for doing this (with memo) using the transfer-client tool is

transfer-client \
  --network=testnet \
  --sender=3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi \
  --receiver=4Gaw3Y44fyGzaNbG69eZyr1Q5fByMvSuQ5pKRW7xRmDzajKtMS \
  --amount=1000 \
  --memo-hex='674869204d6f6d21' \
  --keys-file=./sender.keys

The request/response flow of the command is a sequence of calls to the Construction API.

  1. The derive endpoint derives an account address for a public key. This is not applicable to Concordium.

  2. Call preprocess with a list of operations representing the transfer.

    Request:

    {
 "network_identifier": { "blockchain": "concordium", "network": "testnet" },
 "operations": [
   {
     "operation_identifier": { "index": 0 },
     "type": "transfer",
     "account": { "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi" },
     "amount": {
       "value": "-1000",
       "currency": { "symbol": "CCD", "decimals": 6 }
     }
   },
   {
     "operation_identifier": { "index": 1 },
     "type": "transfer",
     "account": { "address": "4Gaw3Y44fyGzaNbG69eZyr1Q5fByMvSuQ5pKRW7xRmDzajKtMS" },
     "amount": {
       "value": "1000",
       "currency": { "symbol": "CCD", "decimals": 6 }
     }
   }
 ]
}

    Response:

    {
 "options": {
   "sender": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi"
 },
 "required_public_keys": [
   {
     "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi"
   }
 ]
}

  3. Call metadata with the options from the preprocess response to resolve the sender's nonce. This might as well have been the first step as these options are trivially constructed by hand.

    Request:

    {
 "network_identifier": { "blockchain": "concordium", "network": "testnet" },
 "options": {
   "sender": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi"
 }
}

    Response:

    {
 "metadata": {
   "account_nonce": 87
 }
}

  4. Call payloads to construct the transaction. The memo is passed as part of the metadata along with account_nonce obtained from the previous call as well as expiry time and signature count.

    Request:

    {
 "network_identifier": { "blockchain": "concordium", "network": "testnet" },
 "operations": [
   {
     "operation_identifier": { "index": 0 },
     "type": "transfer",
     "account": { "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi" },
     "amount": {
       "value": "-1000",
       "currency": { "symbol": "CCD", "decimals": 6 }
     }
   },
   {
     "operation_identifier": { "index": 1 },
     "type": "transfer",
     "account": { "address": "4Gaw3Y44fyGzaNbG69eZyr1Q5fByMvSuQ5pKRW7xRmDzajKtMS" },
     "amount": {
       "value": "1000",
       "currency": { "symbol": "CCD", "decimals": 6 }
     }
   }
 ],
 "metadata": {
   "account_nonce": 87,
   "expiry_unix_millis": 1648481235675,
   "memo": "674869204d6f6d21",
   "signature_count": 2
 }
}

    Response:

    {
 "unsigned_transaction": "{\"header\":{\"sender\":\"3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi\",\"nonce\":87,\"energyAmount\":611,\"payloadSize\":51,\"expiry\":1648481235},\"payload\":\"16ae79db76ee0f8d93e47a0fc09b8c1ec89ce3932e66ecb351341e3f6e570225180008674869204d6f6d2100000000000003e8\"}",
 "payloads": [
   {
     "account_identifier": { "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi" },
     "hex_bytes": "6b0f407bece7b782998547e3ae4ed4e7df9faa3b621f0d1ed4f0ddaea20a9cbc",
     "signature_type": "ed25519"
   }
 ]
}

  5. Call parse to verify that the constructed transaction match the intended operations.

    Request:

    {
 "network_identifier": { "blockchain": "concordium", "network": "testnet" },
 "signed": false,
 "transaction": "{\"header\":{\"sender\":\"3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi\",\"nonce\":87,\"energyAmount\":611,\"payloadSize\":51,\"expiry\":1648481235},\"payload\":\"16ae79db76ee0f8d93e47a0fc09b8c1ec89ce3932e66ecb351341e3f6e570225180008674869204d6f6d2100000000000003e8\"}"
}

    Response:

    {
 "operations": [
   {
     "operation_identifier": { "index": 0 },
     "type": "transfer",
     "account": { "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi" },
     "amount": {
       "value": "-1000",
       "currency": { "symbol": "CCD", "decimals": 6 }
     }
   },
   {
     "operation_identifier": { "index": 1 },
     "type": "transfer",
     "account": {
       "address": "4Gaw3Y44fyGzaNbG69eZyr1Q5fByMvSuQ5pKRW7xRmDzajKtMS"
     },
     "amount": {
       "value": "1000",
       "currency": { "symbol": "CCD", "decimals": 6 }
     }
   }
 ],
 "metadata": {
   "memo": "674869204d6f6d21"
 }
}

  6. Sign the payloads and call combine with the resulting signatures prepended with the credential/key indexes of the signatures' keys. The server returns an object containing both the transaction and signatures.

    Request:

    {
 "network_identifier": { "blockchain": "concordium", "network": "testnet" },
 "unsigned_transaction": "{\"header\":{\"sender\":\"3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi\",\"nonce\":87,\"energyAmount\":611,\"payloadSize\":51,\"expiry\":1648481235},\"payload\":\"16ae79db76ee0f8d93e47a0fc09b8c1ec89ce3932e66ecb351341e3f6e570225180008674869204d6f6d2100000000000003e8\"}",
 "signatures": [
   {
     "signing_payload": {
       "account_identifier": {
         "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi"
       },
       "hex_bytes": "6b0f407bece7b782998547e3ae4ed4e7df9faa3b621f0d1ed4f0ddaea20a9cbc",
       "signature_type": "ed25519"
     },
     "public_key": {
       "hex_bytes": "660095bfc536effbfdc5bc6ed58ae10810103482ea9e4af02cb5a393c21d8fc6",
       "curve_type": "edwards25519"
     },
     "signature_type": "ed25519",
     "hex_bytes": "0:0/2b31e8dddc617b780db49fc7e1b15da7545082ace496548c1c2eaa97d1628e23d04e04cfa8fd58a500df955192b5aa7ab3e4039900c929bcea71a2bc4bbf3001"
   },
   {
     "signing_payload": {
       "account_identifier": {
         "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi"
       },
       "hex_bytes": "6b0f407bece7b782998547e3ae4ed4e7df9faa3b621f0d1ed4f0ddaea20a9cbc",
       "signature_type": "ed25519"
     },
     "public_key": {
       "hex_bytes": "8de8ff2a9ee861ec64db65d552a59b01bbfc41d51796c6678934ecfb518a2194",
       "curve_type": "edwards25519"
     },
     "signature_type": "ed25519",
     "hex_bytes": "0:1/085c7edb5fb460290c243955e3680070e3279fc82a2fbf6f219352da1ea0f5b773813f48e382d7dd25d1fdb54d262239172409506215e6524500809f6b4bc30f"
   }
 ]
}

    Response:

    {
 "signed_transaction": "{\"signature\":{\"0\":{\"0\":\"2b31e8dddc617b780db49fc7e1b15da7545082ace496548c1c2eaa97d1628e23d04e04cfa8fd58a500df955192b5aa7ab3e4039900c929bcea71a2bc4bbf3001\",\"1\":\"085c7edb5fb460290c243955e3680070e3279fc82a2fbf6f219352da1ea0f5b773813f48e382d7dd25d1fdb54d262239172409506215e6524500809f6b4bc30f\"}},\"header\":{\"sender\":\"3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi\",\"nonce\":87,\"energyAmount\":611,\"payloadSize\":51,\"expiry\":1648481235},\"payload\":\"16ae79db76ee0f8d93e47a0fc09b8c1ec89ce3932e66ecb351341e3f6e570225180008674869204d6f6d2100000000000003e8\"}"
}

  7. Call parse to verify that the signed transaction still match the original operations.

    Request:

    {
 "network_identifier": { "blockchain": "concordium", "network": "testnet" },
 "signed": true,
 "transaction": "{\"signature\":{\"0\":{\"0\":\"2b31e8dddc617b780db49fc7e1b15da7545082ace496548c1c2eaa97d1628e23d04e04cfa8fd58a500df955192b5aa7ab3e4039900c929bcea71a2bc4bbf3001\",\"1\":\"085c7edb5fb460290c243955e3680070e3279fc82a2fbf6f219352da1ea0f5b773813f48e382d7dd25d1fdb54d262239172409506215e6524500809f6b4bc30f\"}},\"header\":{\"sender\":\"3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi\",\"nonce\":87,\"energyAmount\":611,\"payloadSize\":51,\"expiry\":1648481235},\"payload\":\"16ae79db76ee0f8d93e47a0fc09b8c1ec89ce3932e66ecb351341e3f6e570225180008674869204d6f6d2100000000000003e8\"}"
}

    Response:

    {
 "operations": [
   {
     "operation_identifier": { "index": 0 },
     "type": "transfer",
     "account": { "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi" },
     "amount": {
       "value": "-1000",
       "currency": { "symbol": "CCD", "decimals": 6 }
     }
   },
   {
     "operation_identifier": { "index": 1 },
     "type": "transfer",
     "account": { "address": "4Gaw3Y44fyGzaNbG69eZyr1Q5fByMvSuQ5pKRW7xRmDzajKtMS" },
     "amount": {
       "value": "1000",
       "currency": { "symbol": "CCD", "decimals": 6 }
     }
   }
 ],
 "account_identifier_signers": [
   {
     "address": "3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi"
   }
 ],
 "metadata": {
   "memo": "674869204d6f6d21"
 }
}

  8. Call submit to send the transaction to the node that the server is connected to.

    Request:

    {
 "network_identifier": { "blockchain": "concordium", "network": "testnet" },
 "signed_transaction": "{\"signature\":{\"0\":{\"0\":\"2b31e8dddc617b780db49fc7e1b15da7545082ace496548c1c2eaa97d1628e23d04e04cfa8fd58a500df955192b5aa7ab3e4039900c929bcea71a2bc4bbf3001\",\"1\":\"085c7edb5fb460290c243955e3680070e3279fc82a2fbf6f219352da1ea0f5b773813f48e382d7dd25d1fdb54d262239172409506215e6524500809f6b4bc30f\"}},\"header\":{\"sender\":\"3rsc7HNLVKnFz9vmKkAaEMVpNkFA4hZxJpZinCtUTJbBh58yYi\",\"nonce\":87,\"energyAmount\":611,\"payloadSize\":51,\"expiry\":1648481235},\"payload\":\"16ae79db76ee0f8d93e47a0fc09b8c1ec89ce3932e66ecb351341e3f6e570225180008674869204d6f6d2100000000000003e8\"}"
}

    Response:

    {
 "transaction_identifier": {
   "hash": "bea16341103d332d7ff57bde96276722bf7a97b79fbf8a8df0d3711f81f533ef"
 }
}

  9. The hash may be recomputed later (or before) with the hash endpoint, which is just a dry-run variant of submit.

Failure handling

Rosetta doesn't check most causes of transactions being invalid; i.e. things like nonexistent accounts, bad signatures, and insufficient funds. As long as the transaction is "well-formed", Rosetta will accept the transaction and return its hash without complaints.

An exception to this is if the sender account doesn't exist. Then the nonce lookup will fail and result in an error.

Depending on the situation, a submitted invalid transaction may or may not ever get included in a block. Generally speaking, if the transaction is signed correctly and the sender is able to pay a fee, then the transaction will be applied in a failed state. Otherwise it is silently rejected.

For example, if the wrong keys are provided, then the transaction will silently disappear. If the receiver doesn't exist or the sender has insufficient funds, then the only outcome of the transaction is an error message (and the deduction of a fee).

The bottom line is that the only way to confirm that a transaction is successfully applied is to check the hash against the chain. Also, the block containing the transaction has to be finalized for the transaction to be as well.

Testing

Rosetta CLI tool

The Rosetta team maintains a CLI tool that includes commands for verifying that the implementation produces valid results. This includes consistency checks of the balance of all accounts, i.e. that all changes in balances are accounted for in transactions.

The test will fail if run with the official Rosetta CLI tool because it doesn't understand how Concordium does account aliases. We therefore forked the tool to make it accept that a transaction affecting the balance of an account affects all aliases of that account as well.

To run the test you need a running instance of both the concordium-node and the Conocordium Rosetta.

The easiest way to run Rosetta and the tool is by using the provided Docker Compose deployment with the profile check-data enabled. See the following sections for alternative methods.

Build and run (direct)

concordium-rosetta --network testnet

To install the rosetta-cli tool that can run tests follow the steps below:

# Clone our Rosetta-CLI fork
git clone https://github.com/Concordium/rosetta-cli

cd rosetta-cli

# Build the binary
go build .

The default config file can be generated like this:

# Create the config file
cd ./bin
./rosetta-cli configuration:create ./config.json

We need to make the following changes to this configuration:

Now the test tool can be run:

# Check the correctness of a Rosetta Data API Implementation
./rosetta-cli --configuration-file ./config.json check:data

Note that this only tests the data returned by the Rosetta API implementation is valid. It does not test interaction on chain, such as transactions. We test that with a different tool. There is more info on the Rosetta-API website.

Build and run (Docker)

You can also build using the provided docker file in tools/rosetta-cli-docker

It uses the default configuration with the following changes added:

Transfer client

The transfer-client tool (used in example above) is a simple client that uses the Rosetta implementation to make a CCD transfer from one account to another. The transfer may optionally include a memo.

Resources