search

near / near-indexer-for-explorer

Watch NEAR network and store all the data from NEAR blockchain to PostgreSQL database
https://near-indexers.io/docs/projects/near-indexer-for-explorer
GNU General Public License v3.0
123 stars 56 forks source link

Indexer node in archive mode cant download headers #261

Closed teke97 closed 2 years ago

teke97 commented 2 years ago

I'm trying to sync an indexer in archive mode. Node takes a long time to download headers. (minimum 7 days)

Version: indexer-explorer 0.10.11

Server: Intel® Xeon® W-2295 18-core Cascade-Lake W, 128 GB DDR4 ECC RAM, SAMSUNG MZ7LH3T8 SSD

I started by backing up from here

Could you recommend something?

near_indexer.service ```ini [Unit] Description=Near Indexer Wants=network-online.target After=network-online.target [Service] WorkingDirectory=/home/near-indexer User=near-indexer Group=near-indexer Type=simple ExecStart=/usr/local/bin/indexer-explorer --home-dir /home/near-indexer run --store-genesis --concurrency 1 sync-from-interruption [Install] WantedBy=multi-user.target ```
config.json ```json { "genesis_file": "genesis.json", "genesis_records_file": null, "validator_key_file": "validator_key.json", "node_key_file": "node_key.json", "rpc": { "addr": "0.0.0.0:3030", "prometheus_addr": null, "cors_allowed_origins": [ "*" ], "polling_config": { "polling_interval": { "secs": 0, "nanos": 500000000 }, "polling_timeout": { "secs": 10, "nanos": 0 } }, "limits_config": { "json_payload_max_size": 10485760 } }, "telemetry": { "endpoints": [ "https://explorer.mainnet.near.org/api/nodes", "https://explorer.mainnet.near.org/api/nodes" ] }, "network": { "addr": "0.0.0.0:24567", "external_address": "", "boot_nodes": "ed25519:86EtEy7epneKyrcJwSWP7zsisTkfDRH5CFVszt4qiQYw@35.195.32.249:24567,ed25519:BFB78VTDBBfCY4jCP99zWxhXUcFAZqR22oSx2KEr8UM1@35.229.222.235:24567,ed25519:Cw1YyiX9cybvz3yZcbYdG7oDV6D7Eihdfc8eM1e1KKoh@35.195.27.104:24567,ed25519:33g3PZRdDvzdRpRpFRZLyscJdbMxUA3j3Rf2ktSYwwF8@34.94.132.112:24567,ed25519:CDQFcD9bHUWdc31rDfRi4ZrJczxg8derCzybcac142tK@35.196.209.192:24567", "max_num_peers": 40, "minimum_outbound_peers": 5, "ideal_connections_lo": 30, "ideal_connections_hi": 35, "peer_recent_time_window": { "secs": 600, "nanos": 0 }, "safe_set_size": 20, "archival_peer_connections_lower_bound": 10, "handshake_timeout": { "secs": 20, "nanos": 0 }, "reconnect_delay": { "secs": 60, "nanos": 0 }, "skip_sync_wait": false, "ban_window": { "secs": 10800, "nanos": 0 }, "blacklist": [], "ttl_account_id_router": { "secs": 3600, "nanos": 0 }, "peer_stats_period": { "secs": 5, "nanos": 0 } }, "consensus": { "min_num_peers": 3, "block_production_tracking_delay": { "secs": 0, "nanos": 100000000 }, "min_block_production_delay": { "secs": 1, "nanos": 300000000 }, "max_block_production_delay": { "secs": 3, "nanos": 0 }, "max_block_wait_delay": { "secs": 6, "nanos": 0 }, "reduce_wait_for_missing_block": { "secs": 0, "nanos": 100000000 }, "produce_empty_blocks": true, "block_fetch_horizon": 50, "state_fetch_horizon": 5, "block_header_fetch_horizon": 50, "catchup_step_period": { "secs": 0, "nanos": 100000000 }, "chunk_request_retry_period": { "secs": 0, "nanos": 400000000 }, "header_sync_initial_timeout": { "secs": 10, "nanos": 0 }, "header_sync_progress_timeout": { "secs": 2, "nanos": 0 }, "header_sync_stall_ban_timeout": { "secs": 120, "nanos": 0 }, "state_sync_timeout": { "secs": 60, "nanos": 0 }, "header_sync_expected_height_per_second": 10, "sync_check_period": { "secs": 10, "nanos": 0 }, "sync_step_period": { "secs": 0, "nanos": 10000000 }, "doomslug_step_period": { "secs": 0, "nanos": 100000000 } }, "tracked_accounts": [], "tracked_shards": [ 0 ], "archive": true, "log_summary_style": "colored", "gc_blocks_limit": 2, "view_client_threads": 4, "epoch_sync_enabled": true, "view_client_throttle_period": { "secs": 30, "nanos": 0 }, "trie_viewer_state_size_limit": 50000 } ```
Header download stats ```bash ( cat syslog syslog.1; zcat syslog.*.gz ) | grep headers | tac Feb 26 23:43:43 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mFeb 26 23:43:43.406#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 94.31% (2875801)#033[0m #033[1;37m#033[0m #033[1;36m14/1/40 peers ⬇ 4.0kiB/s ⬆ 43.1kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 52%, Mem: 3.8 GiB#033[0m Feb 26 22:31:08 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mFeb 26 22:31:08.650#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 94.31% (2872226)#033[0m #033[1;37m#033[0m #033[1;36m15/1/40 peers ⬇ 480.0kiB/s ⬆ 670.0kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 55%, Mem: 3.8 GiB#033[0m Feb 26 21:41:45 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mFeb 26 21:41:45.031#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 94.32% (2869806)#033[0m #033[1;37m#033[0m #033[1;36m15/1/40 peers ⬇ 33.5kiB/s ⬆ 657.8kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 59%, Mem: 3.6 GiB#033[0m Feb 26 21:40:50 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mFeb 26 21:40:50.051#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 94.32% (2869762)#033[0m #033[1;37m#033[0m #033[1;36m15/1/40 peers ⬇ 486.3kiB/s ⬆ 676.4kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 58%, Mem: 3.6 GiB#033[0m ... Mar 5 02:21:19 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mMar 05 02:21:19.050#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 93.52% (3299693)#033[0m #033[1;37m#033[0m #033[1;36m12/1/40 peers ⬇ 6.4kiB/s ⬆ 79.1kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 51%, Mem: 4.2 GiB#033[0m Mar 5 02:05:37 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mMar 05 02:05:37.886#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 93.52% (3298925)#033[0m #033[1;37m#033[0m #033[1;36m12/1/40 peers ⬇ 449.9kiB/s ⬆ 511.4kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 50%, Mem: 4.2 GiB#033[0m Mar 5 02:05:24 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mMar 05 02:05:24.096#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 93.52% (3298925)#033[0m #033[1;37m#033[0m #033[1;36m12/1/40 peers ⬇ 448.2kiB/s ⬆ 507.2kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 62%, Mem: 4.2 GiB#033[0m Mar 5 01:21:54 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mMar 05 01:21:54.812#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 93.53% (3296193)#033[0m #033[1;37m#033[0m #033[1;36m12/1/40 peers ⬇ 454.0kiB/s ⬆ 514.1kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 56%, Mem: 4.2 GiB#033[0m Mar 5 00:29:35 near-indexer-and-archival-node-2 indexer-explorer[865]: #033[2mMar 05 00:29:35.712#033[0m #033[32m INFO#033[0m stats: #033[1;33m#57462818 Downloading headers 93.53% (3294344)#033[0m #033[1;37m#033[0m #033[1;36m12/1/40 peers ⬇ 6.4kiB/s ⬆ 511.6kiB/s#033[0m #033[1;32m0.00 bps 0 gas/s#033[0m #033[1;34mCPU: 52%, Mem: 4.2 GiB#033[0m ```
khorolets commented 2 years ago

Sorry for the late reply.

First of all, a newer version of Indexer for Explorer has been released since you've opened the issue, did it resolve your problem?

If not, the way of getting the backups has changed, please refer to this page https://near-nodes.io/validator/validator-bootcamp#get-data-backup

We need to update the README to stop misleading people about the backups

khorolets commented 2 years ago

We need to update the README to stop misleading people about the backups

OK, README is updated with #271

khorolets commented 2 years ago

I'm closing because of a missing reply. Feel free to reopen if the issue persists.