pi-node / instructions

268 stars 193 forks source link

Pi Node stuck at catching up #234

Open alyocl opened 2 years ago

alyocl commented 2 years ago

My pi node was stuck suddenly for about a few days. I tried to remove all blockchain data, restarted the pi node and docker is still the situation. Then I uninstall the pi node and docker then reinstall it. The pi node can start to run but the local block number is stuck at 1 and stuck at catching up with the consensus state. Can anyone figure out the solution?

Screenshot 2022-01-27 at 11 48 34 PM Screenshot 2022-01-27 at 11 48 43 PM
JengOuyang commented 2 years ago

I have same issue with you these days, the Local Block Number stuck at 1 and Consensus State switching between "Catching Up" and "Joining SCP". image image

alyocl commented 2 years ago

Something found on the log. All connection to a peer was dropped and rejected, all slots are taken. Seems this is the problem only CT can be solved.

2022-01-28T18:45:04.131 GC54Y [Ledger INFO] Got consensus: [seq=5795679, prev=482e66, txs=0, ops=0, sv: [ SIGNED@testnet1 txH: e6b203, ct: 1643395559, upgrades: [ ] ]] 2022-01-28T18:45:04.131 GC54Y [Ledger INFO] Close of ledger 5795679 buffered 2022-01-28T18:45:04.704 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 181.43.0.167:31402 rejected because all available slots are taken. 2022-01-28T18:45:04.704 GC54Y

2022-01-28T18:45:14.550 GC54Y [Ledger INFO] Got consensus: [seq=5795681, prev=f81a9c, txs=0, ops=0, sv: [ SIGNED@testnet3 txH: 230bad, ct: 1643395570, upgrades: [ ] ]] 2022-01-28T18:45:14.550 GC54Y [Ledger INFO] Close of ledger 5795681 buffered 2022-01-28T18:45:15.840 GC54Y [Overlay ERROR] TCP: message size unacceptable: 1213486160 while not authenticated 2022-01-28T18:45:15.902 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 14.241.120.142:31402 rejected because all available slots are taken. 2022-01-28T18:45:15.902 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:15.902 GC54Y [Overlay INFO] Dropping peer 14.241.120.142:31402, reason peer rejected 2022-01-28T18:45:16.387 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 39.118.197.31:31402 rejected because all available slots are taken. 2022-01-28T18:45:16.387 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:16.388 GC54Y [Overlay INFO] Dropping peer 39.118.197.31:31402, reason peer rejected 2022-01-28T18:45:18.917 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 218.75.165.165:31402 rejected because all available slots are taken. 2022-01-28T18:45:18.917 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:18.917 GC54Y [Overlay INFO] Dropping peer 218.75.165.165:31402, reason peer rejected 2022-01-28T18:45:21.680 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 1.192.91.111:31402 rejected because all available slots are taken. 2022-01-28T18:45:21.680 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:21.681 GC54Y [Overlay INFO] Dropping peer 1.192.91.111:31402, reason peer rejected 2022-01-28T18:45:21.699 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 220.166.116.173:31402 rejected because all available slots are taken. 2022-01-28T18:45:21.699 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:21.699 GC54Y [Overlay INFO] Dropping peer 220.166.116.173:31402, reason peer rejected 2022-01-28T18:45:22.147 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 1.254.175.27:31402 rejected because all available slots are taken. 2022-01-28T18:45:22.147 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:22.147 GC54Y [Overlay INFO] Dropping peer 1.254.175.27:31402, reason peer rejected 2022-01-28T18:45:27.926 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 118.131.46.12:31402 rejected because all available slots are taken. 2022-01-28T18:45:27.927 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:27.927 GC54Y [Overlay INFO] Dropping peer 118.131.46.12:31402, reason peer rejected 2022-01-28T18:45:28.760 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 67.62.230.218:31402 rejected because all available slots are taken. 2022-01-28T18:45:28.760 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:28.761 GC54Y [Overlay INFO] Dropping peer 67.62.230.218:31402, reason peer rejected 2022-01-28T18:45:31.316 GC54Y [Overlay INFO] Evicting non-preferred outbound peer 120.33.92.119:31402 for preferred peer 161.35.227.224:31402 2022-01-28T18:45:31.316 GC54Y [Overlay INFO] Dropping peer 120.33.92.119:31402, reason preferred peer selected instead 2022-01-28T18:45:31.317 GC54Y [Overlay INFO] Connected to 161.35.227.224:31402 2022-01-28T18:45:31.900 GC54Y [Overlay INFO] Peer 161.35.227.224:31402 dropped us, reason ERR_LOAD (peer rejected) 2022-01-28T18:45:33.153 GC54Y [Overlay INFO] Connected to 161.35.238.87:31402 2022-01-28T18:45:33.259 GC54Y [Overlay INFO] Peer 161.35.238.87:31402 dropped us, reason ERR_LOAD (peer rejected) 2022-01-28T18:45:33.971 GC54Y [Overlay INFO] Connected to 14.108.89.66:31402 2022-01-28T18:45:34.022 GC54Y [Overlay INFO] Non preferred outbound authenticated peer 118.68.157.75:31402 rejected because all available slots are taken. 2022-01-28T18:45:34.022 GC54Y [Overlay INFO] If you wish to allow for more outbound connections, please update your configuration file 2022-01-28T18:45:34.022 GC54Y [Overlay INFO] Dropping peer 118.68.157.75:31402, reason peer rejected 2022-01-28T18:45:34.142 GC54Y [Herder INFO] Quorum information for 5795680 : {"agree":3,"cost":7948,"delayed":0,"disagree":0,"fail_at":2,"hash":"c51e25","lag_ms":13,"ledger":5795680,"missing":0,"phase":"EXTERNALIZE"}

alyocl commented 2 years ago

Any update from CT?

patrick-chung commented 2 years ago

I have the same problem. did you solve it.

solocop commented 2 years ago

I have solved it by update system datetime and just wait for 30min Let try it

patrick-chung commented 2 years ago

Hello,

Thank you for your reply.

Thanks,

Patrick

From: cop @. Sent: Friday, March 18, 2022 2:54 PM To: pi-node/instructions @.> Cc: patrick-chung @.>; Comment @.> Subject: Re: [pi-node/instructions] Pi Node stuck at catching up (Issue #234)

I have solved it by update system datetime and just wait for 30min Let try it

— Reply to this email directly, view it on GitHub https://github.com/pi-node/instructions/issues/234#issuecomment-1072056589 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AJIIOVAREK75BAR2BWXFL6DVAQLATANCNFSM5M6QSU6Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you commented. https://github.com/notifications/beacon/AJIIOVFWM7ZOKDUYJYEY2OTVAQLATA5CNFSM5M6QSU62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOH7TESDI.gif Message ID: @. @.> >

michelferia commented 2 years ago

Hi, same here, my node is stuck at "catching up" but sometimes, it goes to "joining SCP"

2022-03-19_8-39-26

could be a problem with the Docker version? I'm using the last one Docker Destop 4.6.0 (75818)

May I try to use "Remove all blockchain data"? What does this action?

thanks in advance

alyocl commented 2 years ago

Hi, same here, my node is stuck at "catching up" but sometimes, it goes to "joining SCP"

2022-03-19_8-39-26

could be a problem with the Docker version? I'm using the last one Docker Destop 4.6.0 (75818)

May I try to use "Remove all blockchain data"? What does this action?

thanks in advance

yes, you may try. Switch off the node then remove all blockchain data. Then restart the pi app and docker. It should be fine.

michelferia commented 2 years ago

Hi, I'm having those errors, any cloues? thanks

image

michelferia commented 2 years ago

Hi there, My node is still stuck in Catching up any update ? is it normal at this stage? thanks!

image

michelferia commented 2 years ago

Hello,

I would like to know if this state is normal or it's not sync with the blockchain, thanks

image

AaryanKapoor22 commented 2 years ago

anyone fixed this yet im stuck with the same

fritzdothy commented 2 years ago

I can't fix may pi node

On Thu, 19 May 2022, 17:02 MineGamer69, @.***> wrote:

anyone fixed this yet im stuck with the same

— Reply to this email directly, view it on GitHub https://github.com/pi-node/instructions/issues/234#issuecomment-1131434337, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWBUCPUAQFXLBSBIXGWS2FLVKX7RDANCNFSM5M6QSU6Q . You are receiving this because you are subscribed to this thread.Message ID: @.***>

ishyjo commented 8 months ago

I'm having issues as well. here's my main log below. Any assistance is appreciated!

[2024-03-14 12:32:15.903] [error] run: stats: drive.info(): err: Error: disk name invalid and / not found at C:\Users\xxxx\AppData\Local\Programs\pi-network-desktop\resources\app.asar\node_modules\node-os-utils\lib\drive.js:62:17 at processTicksAndRejections (internal/process/task_queues.js:94:5) [2024-03-14 12:32:15.929] [info] horizonStatusCommand: start [2024-03-14 12:32:16.145] [info] horizonStatusCommand: done [2024-03-14 12:32:23.226] [info] run: heartbeat: OK [2024-03-14 12:32:23.226] [info] run: heartbeat: update-check: started: local-docker-image: pinetwork/pi-node-docker:latest new-docker-image: pinetwork/pi-node-docker:latest [2024-03-14 12:32:23.227] [info] run: heartbeat: update-check: done: either already up to date, or not allowed to update, or docker daemon / container is not running

ishyjo commented 8 months ago

Update: another error popped up:

[2024-03-14 12:39:16.012] [error] dockerFlavorSync: command error: Error: Command failed: docker-machine env default 'docker-machine' is not recognized as an internal or external command, operable program or batch file.

at ChildProcess.exithandler (child_process.js:304:12)
at ChildProcess.emit (events.js:223:5)
at ChildProcess.EventEmitter.emit (domain.js:475:20)
at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)

The render log below:

state: 'Catching up', protocolVersion: 15, ledgerNumber: 1, incomingNodes: 0, outgoingNodes: 8, latestProtocolVersion: 15 } [2024-03-14 12:41:37.291] [debug] containerFlavor not null [2024-03-14 12:41:41.328] [debug] Renderer: interval happening... [2024-03-14 12:41:41.329] [debug] isStellarNode: true [2024-03-14 12:41:41.329] [debug] ConsensusState: running [2024-03-14 12:41:41.329] [debug] Renderer: getting SCP info [2024-03-14 12:41:41.330] [debug] getLocalSCPInfo: start [2024-03-14 12:41:41.330] [debug] getLocalSCPInfo: fetching info from localhost... [2024-03-14 12:41:41.330] [debug] stellarHttpCommand: start [2024-03-14 12:41:41.333] [debug] containerFlavor not null [2024-03-14 12:41:41.820] [debug] stellarHttpCommand: successful [2024-03-14 12:41:41.820] [debug] getLocalSCPInfo: done fetching result [2024-03-14 12:41:41.820] [debug] getLocalSCPInfo: fetching peers from localhost... [2024-03-14 12:41:41.821] [debug] stellarHttpCommand: start [2024-03-14 12:41:42.415] [debug] stellarHttpCommand: successful [2024-03-14 12:41:42.415] [debug] getLocalSCPInfo: done fetching result [2024-03-14 12:41:42.416] [debug] getLocalSCPInfo: fetching latest protocol version from testnet... [2024-03-14 12:41:42.467] [debug] SCP info: {

remzicirit commented 8 months ago

I am also getting same error with NODE 4.9 version

[2024-04-07 13:04:08.064] [error] dockerFlavorSync: command error: Error: Command failed: docker-machine env default 'docker-machine' is not recognized as an internal or external command, operable program or batch file.

at ChildProcess.exithandler (node:child_process:430:12)
at ChildProcess.emit (node:events:513:28)
at ChildProcess.emit (node:domain:489:12)
at maybeClose (node:internal/child_process:1091:16)
at Process.onexit (node:internal/child_process:302:5)

[2024-04-07 13:04:09.680] [info] horizonStatusCommand: start [2024-04-07 13:04:09.733] [error] run: stats: drive.info(): err: Error: disk name invalid and / not found at C:\Users\Admin\AppData\Local\Programs\pi-network-desktop\resources\app.asar\dist\main\webpack:\pi-network-desktop\node_modules\node-os-utils\lib\drive.js:62:17 at processTicksAndRejections (node:internal/process/task_queues:95:5)

ishyjo commented 8 months ago

Update: read in some other forum that downgrading Decker Desktop to a previous version helped. I ended up uninstalling latest Desker Desktop and installed v4.20.1 and pi node started working and synced local block chain with latest.

https://desktop.docker.com/win/main/amd64/110738/Docker%20Desktop%20Installer.exe?_gl=1*ldyvmv*_ga*NDU3ODkzMjQ3LjE3MTE4NTgxNTk.*_ga_XJWPQMJYHQ*MTcxMjU3NzgxMy4yLjEuMTcxMjU3ODA5MS41LjAuMA..