search

baking-bad / tzkt

😼 Awesome Tezos blockchain indexer and API
https://tzkt.io
MIT License
183 stars 35 forks source link
blockchain-api blockchain-indexer tezos-api tezos-blockchain tezos-indexer tzkt-api tzkt-indexer

readme

Tezos Indexer by Baking Bad

Made With License: MIT

TzKT is the most advanced Tezos blockchain indexer with powerful API created by the Baking Bad team with huge support from the Tezos Foundation.

The indexer fetches raw data from the Tezos blockchain, processes it, and saves it to its database to provide efficient access to the blockchain data. Using indexers is necessary part for most blockchain-related applications, because indexers expose much more data and cover much more use-cases than native node RPC, for example getting operations by hash, or operations related to particular accounts and smart contracts, or created NFTs, or token balances, or baking rewards, etc.

Features:

Installation (docker)

First of all, install git, make, docker, docker-compose, then run the following commands:

git clone https://github.com/baking-bad/tzkt
cd tzkt/

make init  # Restores DB from the latest snapshot. Skip it, if you want to index from scratch.
make start # Starts DB, indexer, and API. By default, the API will be available at http://127.0.0.1:5000.
make stop  # Stops DB, indexer, and API.

You can configure TzKT via Tzkt.Sync/appsettings.json (indexer) and Tzkt.Api/appsettings.json (API). All the settings can also be passed via env vars or command line args. See an example of how to provide settings via env vars and read some tips about indexer configuration and API configuration.

Installation (from source)

This guide is for Ubuntu 22.04, but even if you use a different OS, the installation process will likely be the same, except for the "Install packages" part.

Install packages

Install Git

sudo apt update
sudo apt install git

Install .NET

wget https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb

sudo apt update
sudo apt install -y dotnet-sdk-7.0

Install Postgresql

sudo sh -c 'echo "deb https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt update
sudo apt -y install postgresql-16 postgresql-contrib-16

Prepare database

Create an empty database and its user

sudo -u postgres psql

postgres=# create database tzkt_db;
postgres=# create user tzkt with encrypted password 'qwerty';
postgres=# grant all privileges on database tzkt_db to tzkt;
postgres=# \q

Download fresh snapshot (example for mainnet)

wget "https://snapshots.tzkt.io/tzkt_v1.14_mainnet.backup" -O /tmp/tzkt_db.backup

Restore database from the snapshot

sudo -u postgres pg_restore -c --if-exists -v -1 -d tzkt_db /tmp/tzkt_db.backup

Notes:

Grant access to the database to our user

sudo -u postgres psql tzkt_db

tzkt_db=# grant all privileges on all tables in schema public to tzkt;
tzkt_db=# \q

Build, configure and run TzKT Indexer

Clone repo

git clone https://github.com/baking-bad/tzkt.git ~/tzkt

Build indexer

cd ~/tzkt/Tzkt.Sync/
dotnet publish -o ~/tzkt-sync

Configure indexer (example for mainnet)

Edit the configuration file ~/tzkt-sync/appsettings.json. What you basically need is to adjust the TezosNode.Endpoint and ConnectionStrings.DefaultConnection, if needed:

{
  "TezosNode": {
    "Endpoint": "https://rpc.tzkt.io/mainnet/"
  },
  "ConnectionStrings": {
    "DefaultConnection": "host=localhost;port=5432;database=tzkt_db;username=tzkt;password=qwerty;command timeout=600;"
  }
}

Read more about connection string and available parameters.

Chain reorgs and indexing lag

To avoid reorgs (chain reorganizations) you can set the indexing lag TezosNode.Lag (1-2 blocks lag is enough):

{
  "TezosNode": {
    "Lag": 1
  }
}
Collect metrics

You can enable/disable Prometheus metrics by setting MetricsOptions.Enabled. By default, they will be available at http://localhost:5001/metrics (protobuf) and http://localhost:5001/metrics-text (plain text):

  "MetricsOptions": {
    "Enabled": true
  }

Run indexer

cd ~/tzkt-sync
dotnet Tzkt.Sync.dll

That's it. If you want to run the indexer as a daemon, take a look at this guide: https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/linux-nginx?view=aspnetcore-7.0#create-the-service-file.

Build, configure and run TzKT API

Suppose, you have already cloned the repo to ~/tzkt during the steps above.

Build API

cd ~/tzkt/Tzkt.Api/
dotnet publish -o ~/tzkt-api

Configure API

Edit the configuration file ~/tzkt-api/appsettings.json. What you basically need is to adjust the ConnectionStrings.DefaultConnection, if needed:

Like this:

{
  "ConnectionStrings": {
    "DefaultConnection": "host=localhost;port=5432;database=tzkt_db;username=tzkt;password=qwerty;command timeout=600;"
  },
}

Read more about connection string and available parameters.

Response cache

The API has built-in response cache, enabled by default. You can control the cache size limit by setting the ResponseCache.CacheSize (MB), or disable it by setting to 0:

{
   "ResponseCache": {
      "CacheSize": 1024
   }
}
RPC helpers (example for mainnet)

The API provides RPC helpers - endpoints proxied directly to the node RPC, specified in the API settings. The Rpc helpers can be enabled in the RpcHelpers section:

{
   "RpcHelpers": {
      "Enabled": true,
      "Endpoint": "https://rpc.tzkt.io/mainnet/"
   }
}

Please, notice, the API's RpcHelpers.Endpoint must point to the same network (with the same chain_id) as TezosNode.Endpoint in the indexer. Otherwise, an exception will be thrown.

Collect metrics

You can enable/disable Prometheus metrics by setting MetricsOptions.Enabled. By default, they will be available at http://localhost:5000/metrics (protobuf) and http://localhost:5000/metrics-text (plain text):

  "MetricsOptions": {
    "Enabled": true
  }
TCP port

By default, the API is available at the port 5000. You can configure it at Kestrel.Endpoints.Http.Url:

  "Kestrel": {
    "Endpoints": {
      "Http": {
        "Url": "http://localhost:5000"
      }
    }
  }

Run API

cd ~/tzkt-api
dotnet Tzkt.Api.dll

That's it. If you want to run the API as a daemon, take a look at this guide: https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/linux-nginx?view=aspnetcore-7.0#create-the-service-file.

Install Tzkt Indexer and API for testnets

In general the steps are the same as for the mainnet, you will just need to use a different RPC endpoint and DB snapshot. Here are presets for the current testnets:

Testnets & docker

First of all, install git, make, docker, docker-compose, then run the following commands:

git clone https://github.com/baking-bad/tzkt.git
cd tzkt/

make ghost-init  # Restores DB from the latest snapshot. Skip it, if you want to index from scratch.
make ghost-start # Starts DB, indexer, and API. By default, the API will be available at http://127.0.0.1:5010.
make ghost-stop  # Stops DB, indexer, and API.

Have a question?

Feel free to contact us via:

Cheers! 🍺