search

threefoldtech / grid_deployment

Deploy a full Grid backend with docker-compose and snapshots
Apache License 2.0
2 stars 0 forks source link

[tfchain val] test a sync with new prune flags #71

Closed coesensbert closed 2 months ago

coesensbert commented 2 months ago

https://github.com/threefoldtech/grid_deployment/commit/e4de06b7c9ece11da35c57805e9e843995174485

coesensbert commented 2 months ago

on a node that is setup correctly, I'm not able to find peers on devnet to sync

root@zos-tst ~/grid_deployment/tfchain-validator/devnet # docker logs tfchain-validator 
2024-09-04 14:03:34.080  INFO main sc_cli::runner: Substrate Node    
2024-09-04 14:03:34.081  INFO main sc_cli::runner: āœŒļø  version 2.7.0-rc4-unknown    
2024-09-04 14:03:34.081  INFO main sc_cli::runner: ā¤ļø  by dylan@threefold.tech, 2017-2024    
2024-09-04 14:03:34.081  INFO main sc_cli::runner: šŸ“‹ Chain specification: TF Chain Devnet    
2024-09-04 14:03:34.081  INFO main sc_cli::runner: šŸ·  Node name: dev-val-sync-tst    
2024-09-04 14:03:34.081  INFO main sc_cli::runner: šŸ‘¤ Role: AUTHORITY    
2024-09-04 14:03:34.081  INFO main sc_cli::runner: šŸ’¾ Database: RocksDb at /storage/chains/tfchain_devnet/db/full    
2024-09-04 14:03:34.275  WARN main sc_service::config: Using default protocol ID "sup" because none is configured in the chain specs    
2024-09-04 14:03:34.275  INFO main sub-libp2p: šŸ·  Local node identity is: 12D3KooWHe3EhmkiBQExosLMknMUzidiRLYifyNsAecdM1WyY8JL    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» Operating system: linux    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» CPU architecture: x86_64    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» Target environment: gnu    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» CPU: 12th Gen Intel(R) Core(TM) i5-12500    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» CPU cores: 6    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» Memory: 64108MB    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» Kernel: 6.8.0-40-generic    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» Linux distribution: Ubuntu 22.04.3 LTS    
2024-09-04 14:03:34.279  INFO main sc_sysinfo: šŸ’» Virtual machine: no    
2024-09-04 14:03:34.279  INFO main sc_service::builder: šŸ“¦ Highest known block at #0    
2024-09-04 14:03:34.279  INFO                 main sc_rpc_server: Running JSON-RPC server: addr=127.0.0.1:9944, allowed origins=["http://localhost:*", "http://127.0.0.1:*", "https://localhost:*", "https://127.0.0.1:*", "https://polkadot.js.org"]    
2024-09-04 14:03:34.279  INFO tokio-runtime-worker substrate_prometheus_endpoint: ć€½ļø Prometheus exporter started at 0.0.0.0:9615    
2024-09-04 14:03:39.280  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:03:44.281  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:03:49.282  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:03:54.282  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:03:59.283  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:04.284  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:09.284  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:14.285  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:19.285  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:24.286  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:29.287  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:34.287  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:39.288  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:44.289  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0    
2024-09-04 14:04:49.289  INFO tokio-runtime-worker substrate: šŸ’¤ Idle (0 peers), best: #0 (0xbb7fā€¦66be), finalized #0 (0xbb7fā€¦66be), ā¬‡ 0 ā¬† 0
coesensbert commented 2 months ago

sync issue resolved with correct image version (that includes the bootnodes): https://github.com/threefoldtech/grid_deployment/commit/9770d7fb0fa0d239448962d90b9600e19df875b7 Sync test started: https://telemetry.tfchain.grid.tf/#list/0xbb7f298ba5fca8802040061ddd0ca6406832238b7825ebfa66927cc25ec666be

root@zos-tst ~/grid_deployment/tfchain-validator/devnet # docker logs tfchain-validator 
2024-09-04 14:18:32.519  INFO main sc_cli::runner: Substrate Node    
2024-09-04 14:18:32.519  INFO main sc_cli::runner: āœŒļø  version 2.9.0-unknown    
2024-09-04 14:18:32.519  INFO main sc_cli::runner: ā¤ļø  by dylan@threefold.tech, 2017-2024    
2024-09-04 14:18:32.519  INFO main sc_cli::runner: šŸ“‹ Chain specification: TF Chain Devnet    
2024-09-04 14:18:32.519  INFO main sc_cli::runner: šŸ·  Node name: dev-val-sync-tst    
2024-09-04 14:18:32.519  INFO main sc_cli::runner: šŸ‘¤ Role: AUTHORITY    
2024-09-04 14:18:32.519  INFO main sc_cli::runner: šŸ’¾ Database: RocksDb at /storage/chains/tfchain_devnet/db/full    
2024-09-04 14:18:33.088  INFO main sc_service::client::client: šŸ”Ø Initializing Genesis block/state (state: 0x6a2eā€¦4030, header-hash: 0xbb7fā€¦66be)    
2024-09-04 14:18:33.091  INFO main grandpa: šŸ‘“ Loading GRANDPA authority set from genesis on what appears to be first startup.    
2024-09-04 14:18:33.224  WARN main sc_service::config: Using default protocol ID "sup" because none is configured in the chain specs    
2024-09-04 14:18:33.224  INFO main sub-libp2p: šŸ·  Local node identity is: 12D3KooWHe3EhmkiBQExosLMknMUzidiRLYifyNsAecdM1WyY8JL    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» Operating system: linux    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» CPU architecture: x86_64    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» Target environment: gnu    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» CPU: 12th Gen Intel(R) Core(TM) i5-12500    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» CPU cores: 6    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» Memory: 64108MB    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» Kernel: 6.8.0-40-generic    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» Linux distribution: Ubuntu 22.04.4 LTS    
2024-09-04 14:18:33.228  INFO main sc_sysinfo: šŸ’» Virtual machine: no    
2024-09-04 14:18:33.228  INFO main sc_service::builder: šŸ“¦ Highest known block at #0    
2024-09-04 14:18:33.228  INFO tokio-runtime-worker substrate_prometheus_endpoint: ć€½ļø Prometheus exporter started at 0.0.0.0:9615    
2024-09-04 14:18:33.228  INFO                 main sc_rpc_server: Running JSON-RPC server: addr=127.0.0.1:9944, allowed origins=["http://localhost:*", "http://127.0.0.1:*", "https://localhost:*", "https://127.0.0.1:*", "https://polkadot.js.org"]    
2024-09-04 14:18:33.816  INFO tokio-runtime-worker sub-libp2p: šŸ” Discovered new external address for our node: /ip4/144.76.14.155/tcp/30333/p2p/12D3KooWHe3EhmkiBQExosLMknMUzidiRLYifyNsAecdM1WyY8JL    
2024-09-04 14:18:34.268  WARN tokio-runtime-worker peerset: Report 12D3KooWB1WjEfiobvuoqUV9PzdTBFS36Cei7NpkwogzqY6oJ9ba: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:35.227  WARN tokio-runtime-worker peerset: Report 12D3KooWMkj7iVNGcTUgwbZN5i5taQDnyk8oMQ4akAEv3apAYc4Z: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:35.254  WARN tokio-runtime-worker peerset: Report 12D3KooWBPGEVpQcCo7LcYp3PZdTojkub2TQZf4fzisznxtV7XhG: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:35.258  WARN tokio-runtime-worker peerset: Report 12D3KooWK3QKc4HBSFPuHnGth9voyGLkVDFVXSKQSkpr8wD9QXSp: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:35.268  WARN tokio-runtime-worker peerset: Report 12D3KooWSCFtAPQRRa5uw2QhwsC77NfCv1mxHJjWQhUrRANkoYyx: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:35.269  WARN tokio-runtime-worker peerset: Report 12D3KooWE19pLyW7LHkCDQPttrkDtquNt89LLUZtpXayv2Uc8dd2: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:35.270  WARN tokio-runtime-worker peerset: Report 12D3KooWLh6ZJu9NYCVZoX1HFwF52ztk4Pg2ztY8UWvUe1CFSYnf: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:36.226  WARN tokio-runtime-worker peerset: Report 12D3KooWEa179ngCGUH625sALh6KxgzGwva4x68JYs9VSThhkESm: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:36.252  WARN tokio-runtime-worker peerset: Report 12D3KooWG3nvgcM3hYoru19A7C79meUNATFWRhwd7tsyj2huMGnz: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:36.308  WARN tokio-runtime-worker peerset: Report 12D3KooWFGwAsgLYuNhTa5XbSxdNqCLojXPbJRQhtNZAUohmPZfB: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:37.271  WARN tokio-runtime-worker peerset: Report 12D3KooWLkEMTViyC1fzU8FzT9HGQtroZZVUaeGP4JQo2fVF9B22: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:37.560  WARN tokio-runtime-worker peerset: Report 12D3KooWBGXALhTf9vY6WstH2ub3R8uMTY584vkeszuhAXL3CypH: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:38.229  INFO tokio-runtime-worker substrate: āš™ļø  Syncing, target=#12678797 (8 peers), best: #5524 (0xed8cā€¦bf42), finalized #5120 (0x766fā€¦661f), ā¬‡ 403.2kiB/s ā¬† 11.9kiB/s    
2024-09-04 14:18:38.268  WARN tokio-runtime-worker peerset: Report 12D3KooWNuRL8uSKAgDyW1Qx49uTdnbXdzk5i2Z1b5m65Nj6cE7V: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:39.275  WARN tokio-runtime-worker peerset: Report 12D3KooWCxk23wdThnKKrr4aDeusSDDh8NCPYM6RkBi1LgkCTVVX: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:40.227  WARN tokio-runtime-worker peerset: Report 12D3KooWES1jUpjghmZGEKyZfej5WtXAGCPZAxVHw7xGUe5mCssp: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:43.229  INFO tokio-runtime-worker substrate: āš™ļø  Syncing 968.0 bps, target=#12678798 (9 peers), best: #10364 (0xadf8ā€¦0f6c), finalized #10240 (0x310aā€¦6c4e), ā¬‡ 230.0kiB/s ā¬† 1.9kiB/s    
2024-09-04 14:18:44.267  WARN tokio-runtime-worker peerset: Report 12D3KooWJgFf1DciJANgpoVUNe7Zavq6Wkge1obiZXwaGBT7ks4z: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.    
2024-09-04 14:18:45.268  WARN tokio-runtime-worker peerset: Report 12D3KooWKwZcU3S3Wwu1ppgk5FrFcdUVcT3LUZTMsGdxodMVAtT1: -2147483648 to -2147483648. Reason: Genesis mismatch. Banned, disconnecting.
coesensbert commented 2 months ago

Successfully synced a devnet node from 0 with the new pruning flags. Took about 17h on an i5-12500 with nvme ssds. Stored data size is around 13GB, while a public RPC node has 110G. We can lower the storage requirements by a lot