CKB node sync pauses because connected peers’ heights are not the latest

sunchengzhu commented 2 weeks ago

Bug Report

Current Behavior

I am a Neuron wallet user and have been maintaining a CKB full node for a long time, meaning I regularly start the CKB node via Neuron to sync the block height. I’ve noticed that the block height stays at the latest height of the connected peers, but their heights are not the most recent. Restarting does not resolve this issue. Does CKB have a mechanism to ensure I connect to nodes with the latest heights for syncing data?

curl -s -X POST 127.0.0.1:8114  -H 'Content-Type: application/json' -d '{ "id": 42, "jsonrpc": "2.0", "method": "sync_state", "params": [] }' | jq
curl -s -X POST 127.0.0.1:8114  -H 'Content-Type: application/json' -d '{ "id": 42, "jsonrpc": "2.0", "method": "get_peers", "params": [] }' | jq | grep last_common_header_number

{
  "jsonrpc": "2.0",
  "result": {
    "assume_valid_target": "0xca44ae8f7bc12ba8eab3224cbe3156c913e2284693e36dc1d01e4d30f362f3c2",
    "assume_valid_target_reached": true,
    "best_known_block_number": "0xde240b",
    "best_known_block_timestamp": "0x19329cd04a1",
    "fast_time": "0x3e8",
    "ibd": true,
    "inflight_blocks_count": "0x0",
    "low_time": "0x5dc",
    "min_chain_work": "0x3314412053c82802a7",
    "min_chain_work_reached": true,
    "normal_time": "0x4e2",
    "orphan_blocks_count": "0x1f",
    "orphan_blocks_size": "0x15d22"
  },
  "id": 42
}

        "last_common_header_number": "0xdba471",
        "last_common_header_number": "0xdba49e",
        "last_common_header_number": "0xdba471",
        "last_common_header_number": "0xdba471",
        "last_common_header_number": "0xdba471",
        "last_common_header_number": "0xdba471",
        "last_common_header_number": "0xdba471",
        "last_common_header_number": "0xdba471",

./ckb-cli rpc get_tip_header --output-format json | jq -r '.number'
14394526

14394526 = 0xdba49e

Expected Behavior

CKB node syncs to the latest, thus making Neuron usable.

Environment

CKB version:ckb 0.117.0 (4920ecf 2024-07-29)
Chain: mainnet
Operating system: macOS 15.0.1
Arch: aarch64
Installation: GitHub Release
Additional context/Screenshots

addr_manager.txt

The DB file could not be uploaded to the issue, so it was renamed with a .txt suffix.

sunchengzhu commented 1 week ago

I found that deleting the data/network directory and then restarting CKB still does not resolve the issue.

eval-exec commented 1 week ago

Could you provide the RPC responses for sync_state and get_peers again?

sunchengzhu commented 1 week ago

@eval-exec Hello, I have been in contact with the developers of Neuron, and they informed me that using the Internal Node in Neuron would utilize the built-in version (0.117.0) of CKB. However, the CKB binary in my CKB directory has already been updated to 0.119.0. I have launched CKB nodes both directly using./ckb run and through starting CKB with Neuron, sharing the same data directory. This means I have synchronized data using different versions of CKB. Could this method of syncing CKB be causing this issue?

sunchengzhu@sunchengzhudeMac-Studio ckb_v0.116.1_aarch64-apple-darwin % find . -name "ckb"
./ckb
sunchengzhu@sunchengzhudeMac-Studio ckb_v0.116.1_aarch64-apple-darwin % cat data/logs/run.log|grep "ckb version"
2024-11-25 08:26:05.660 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.119.0 (69d4df8 2024-10-25)
2024-11-25 08:40:15.733 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.117.0 (4920ecf 2024-07-29)
2024-11-25 08:40:45.932 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.117.0 (4920ecf 2024-07-29)
2024-11-25 08:41:15.934 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.117.0 (4920ecf 2024-07-29)
2024-11-25 08:41:21.217 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.119.0 (69d4df8 2024-10-25)
2024-11-25 08:41:29.194 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.119.0 (69d4df8 2024-10-25)
2024-11-25 08:41:38.246 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.119.0 (69d4df8 2024-10-25)
2024-11-25 08:42:09.599 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.119.0 (69d4df8 2024-10-25)
2024-11-25 08:44:59.677 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.119.0 (69d4df8 2024-10-25)
2024-11-25 08:48:21.336 +00:00 main INFO ckb_bin::subcommand::run  ckb version: 0.117.0 (4920ecf 2024-07-29)
sunchengzhu@sunchengzhudeMac-Studio ckb_v0.116.1_aarch64-apple-darwin % ./ckb --version                         
ckb 0.119.0 (69d4df8 2024-10-25)

eval-exec commented 1 week ago

@eval-exec Hello, I have been in contact with the developers of Neuron, and they informed me that using the Internal Node in Neuron would utilize the built-in version (0.117.0) of CKB. However, the CKB binary in my CKB directory has already been updated to 0.119.0. I have launched CKB nodes both directly using./ckb run and through starting CKB with Neuron, sharing the same data directory. This means I have synchronized data using different versions of CKB. Could this method of syncing CKB be causing this issue?

Yes, to downgrade CKB, you should first use version 0.119.0 to sync the CKB data to the latest state before continuing with a lower version like 0.117.0.

nervosnetwork / ckb