toncenter / ton-indexer

TON Indexer system to store and serve blockchain data using SQL-database
https://toncenter.com/api/v3/
GNU General Public License v3.0
69 stars 34 forks source link

Failed to fetch seqno : Error : 651 : seqno not in db #71

Closed vataops closed 4 months ago

vataops commented 4 months ago
[ 3][t 1][2024-07-12 02:09:48.229597210][InsertManagerBase.cpp:18][!insertmanager]  InsertManagerBase::start_up called
[ 3][t 1][2024-07-12 02:09:48.229693085][InsertManagerBase.cpp:29][!insertmanager]  Insert manager(parallel=12, max_batch_mc_blocks=2000, max_batch_blocks=2000, max_batch_txs=10000, max_batch_msgs=10000)
[ 3][t 1][2024-07-12 02:09:48.229713986][InsertManagerPostgres.cpp:1139][!insertmanager]    Reading existing seqnos
[ 3][t 1][2024-07-12 02:09:48.251325617][IndexScheduler.cpp:76][!indexscheduler]    Found 0 existing seqnos
[ 1][t 4][2024-07-12 01:36:58.359039148][IndexScheduler.cpp:104][!indexquery]   Failed to fetch seqno 405087: [Error : 651 : seqno not in db]
[ 2][t 4][2024-07-12 01:36:58.359047263][IndexScheduler.cpp:115][!indexscheduler]   Rescheduling seqno 405087
[ 1][t 4][2024-07-12 01:36:58.359059982][IndexScheduler.cpp:104][!indexquery]   Failed to fetch seqno 405088: [Error : 651 : seqno not in db]
[ 2][t 4][2024-07-12 01:36:58.359065085][IndexScheduler.cpp:115][!indexscheduler]   Rescheduling seqno 405088
[ 1][t 4][2024-07-12 01:36:58.359069892][IndexScheduler.cpp:104][!indexquery]   Failed to fetch seqno 405089: [Error : 651 : seqno not in db]
[ 2][t 4][2024-07-12 01:36:58.359074593][IndexScheduler.cpp:115][!indexscheduler]   Rescheduling seqno 405089
[ 1][t 4][2024-07-12 01:36:58.359079738][IndexScheduler.cpp:104][!indexquery]   Failed to fetch seqno 405090: [Error : 651 : seqno not in db]
[ 2][t 4][2024-07-12 01:36:58.359084136][IndexScheduler.cpp:115][!indexscheduler]   Rescheduling seqno 405090
[ 1][t 4][2024-07-12 01:36:58.359091744][IndexScheduler.cpp:104][!indexquery]   Failed to fetch seqno 405091: [Error : 651 : seqno not in db]
[ 2][t 4][2024-07-12 01:36:58.359101703][IndexScheduler.cpp:115][!indexscheduler]   Rescheduling seqno 405091
[ 1][t 4][2024-07-12 01:36:58.359109752][IndexScheduler.cpp:104][!indexquery]   Failed to fetch seqno 405092: [Error : 651 : seqno not in db]
[ 2][t 4][2024-07-12 01:36:58.359118443][IndexScheduler.cpp:115][!indexscheduler]   Rescheduling seqno 405092
[ 1][t 7][2024-07-12 01:36:59.359984421][IndexScheduler.cpp:104][!indexquery]   Failed to fetch seqno 405100: [Error : 651 : seqno not in db]
[ 2][t 7][2024-07-12 01:36:59.360019306][IndexScheduler.cpp:115][!indexscheduler]   Rescheduling seqno 405100

I'm running a TON full node and trying to set up the TON indexer, but I'm encountering errors at the worker. With the following directory structure, I've configured the DB and other settings in the .env file like this. Where could the problem be?

POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_USER=postgres
POSTGRES_PASSWORD_FILE=private/postgres_password
POSTGRES_DBNAME=ton_index
POSTGRES_PUBLISH_PORT=5432

TON_INDEXER_API_ROOT_PATH=
TON_INDEXER_API_PORT=8081
TON_INDEXER_API_TITLE=TON_Indexer
TON_INDEXER_WORKERS=1

TON_INDEXER_TON_HTTP_API_ENDPOINT=

TON_WORKER_DBROOT=/mnt/ton/ton-work/db
TON_WORKER_FROM=3
TON_WORKER_MAX_PARALLEL_TASKS=16
TON_WORKER_INSERT_BATCH_SIZE=512
TON_WORKER_INSERT_PARALLEL_ACTORS=3
root@atn-ton-mainnet-full4-vultr-sg:/mnt/ton# tree -L 2
.
├── archive
│   ├── packages
│   ├── states
│   └── tmp
├── files
│   └── packages
└── ton-work
    ├── archive
    ├── db
    ├── files
    ├── keys
dungeon-master-666 commented 4 months ago

In your .env file you have TON_WORKER_FROM=3 meaning that indexer will try to index the whole blockchain from the very beginning. It will work only in case you have an archival node. If you have a regular node you should set this variable to the block that exists in your node db (regular node has 2 weeks retention period).

vataops commented 4 months ago

In your .env file you have TON_WORKER_FROM=3 meaning that indexer will try to index the whole blockchain from the very beginning. It will work only in case you have an archival node. If you have a regular node you should set this variable to the block that exists in your node db (regular node has 2 weeks retention period).

I'm currently running a full node and I want to connect an indexer to it. Does that mean I just need to input the block number where the value TON_WORKER_FROM has been stored for 2 weeks?

dungeon-master-666 commented 4 months ago

Does that mean I just need to input the block number where the value TON_WORKER_FROM has been stored for 2 weeks?

Yes, it will help.

vataops commented 4 months ago

Does that mean I just need to input the block number where the value TON_WORKER_FROM has been stored for 2 weeks?

Yes, it will help.

Is it okay for multiple index-workers to connect to a single PostgreSQL database? For example, I want to know if workers have a feature like Upsert