search

MinaProtocol / mina

Mina is a cryptocurrency protocol with a constant size blockchain, improving scaling while maintaining decentralization and security.
https://minaprotocol.com
Apache License 2.0
1.99k stars 529 forks source link

[Error] Dropping blocks because libp2p validation expired #9726

Open hsq125 opened 3 years ago

hsq125 commented 3 years ago

Preliminary Checks

Description

I'm reporting this specific issue because I see that there is not ticket dedicated to this error, altough it is there for a long time.

[Error] Dropping blocks because libp2p validation expired

With a bit of context ...

Oct 27 23:09:23 ip-172-31-80-92 mina[704]: 2021-10-27 23:09:23 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
Oct 27 23:09:23 ip-172-31-80-92 mina[704]:         error: {
Oct 27 23:09:23 ip-172-31-80-92 mina[704]:   "commit_id": "fe51f1ef5cfbcb4d5f559af39080963cf28ff1af",
Oct 27 23:09:23 ip-172-31-80-92 mina[704]:   "string":
Oct 27 23:09:23 ip-172-31-80-92 mina[704]:     "RPC #2599 failed: \"internal RPC error error: unknown stream_idx\""
Oct 27 23:09:23 ip-172-31-80-92 mina[704]: }
Oct 27 23:09:27 ip-172-31-80-92 mina[704]: 2021-10-27 23:09:27 UTC [Warn] failed to reset stream (this means it was probably closed successfully): $error
Oct 27 23:09:27 ip-172-31-80-92 mina[704]:         error: {
Oct 27 23:09:27 ip-172-31-80-92 mina[704]:   "commit_id": "fe51f1ef5cfbcb4d5f559af39080963cf28ff1af",
Oct 27 23:09:27 ip-172-31-80-92 mina[704]:   "string":
Oct 27 23:09:27 ip-172-31-80-92 mina[704]:     "RPC #2604 failed: \"internal RPC error error: unknown stream_idx\""
Oct 27 23:09:27 ip-172-31-80-92 mina[704]: }
Oct 27 23:09:27 ip-172-31-80-92 mina[704]: 2021-10-27 23:09:27 UTC [Info] Daemon started process of kind "Verifier" with pid 2513
Oct 27 23:09:35 ip-172-31-80-92 mina[704]: 2021-10-27 23:09:35 UTC [Error] verifier terminated unexpectedly
Oct 27 23:09:35 ip-172-31-80-92 mina[704]: 2021-10-27 23:09:35 UTC [Info] Starting a new verifier process
Oct 27 23:09:35 ip-172-31-80-92 mina[704]: 2021-10-27 23:09:35 UTC [Info] verifier successfully stopped
Oct 27 23:10:08 ip-172-31-80-92 mina[704]: 2021-10-27 23:10:08 UTC [Warn] validation callback timed out before we could respond
Oct 27 23:10:08 ip-172-31-80-92 mina[704]: 2021-10-27 23:10:08 UTC [Error] Dropping blocks because libp2p validation expired
Oct 27 23:10:08 ip-172-31-80-92 mina[704]: 2021-10-27 23:10:08 UTC [Error] Duplicate producer and slot: producer = $block_producer, consensus_time = $consensus_time, previous protocol state hash = $hash>
Oct 27 23:10:08 ip-172-31-80-92 mina[704]:         block_producer: "B62qmeVRDf21EkWTy1oahYzn3LXxaGRwPVcY6gTPzMvpHp5ZEtYkArM"
Oct 27 23:10:08 ip-172-31-80-92 mina[704]:         consensus_time: { "slot_number": "107981", "slots_per_epoch": "7140" }
Oct 27 23:10:08 ip-172-31-80-92 mina[704]:         hash: "3NKrrLJG17k8ho9R7gmxtmsptWuX687QiPDbmnRjWPFqYRvwErkA"
Oct 27 23:10:08 ip-172-31-80-92 mina[704]:         current_protocol_state_hash: "3NLfkJjoprhynKQpWaYCaMxu8VY3ZUHmec5ichnff1dgnjRuz3vB"
Oct 27 23:10:08 ip-172-31-80-92 mina[704]: 2021-10-27 23:10:08 UTC [Error] error sending message on stream 1418: $error
Oct 27 23:10:08 ip-172-31-80-92 mina[704]:         error: {

log export in https://s3.amazonaws.com/mina.crowbar.systems/mainnet/export/2021-10-28_07-18-20.tar.gz

Steps to Reproduce

Running 1.2.0 on Ubuntu

Expected Result

Not having this kind of entry in the log

Actual Result

Having this kind of entry in the log

How frequently do you see this issue?

Frequently

What is the impact of this issue on your ability to run a node?

Medium

Status

Mina daemon status
-----------------------------------

Global number of accounts:                     68555
Block height:                                  75165
Max observed block height:                     75165
Max observed unvalidated block height:         75165
Local uptime:                                  6h6m9s
Ledger Merkle root:                            jwJnd2gh8nj2pr9xtZAz3Ty7zWFTrappGuzcAaDKe2Mwh3LagZD
Protocol state hash:                           3NLfVau9AYWhJ3qmLJQYcPu5TFZUvMXS73AozLzfV3Ey9YxSR1vR
Chain id:                                      5f704cc0c82e0ed70e873f0893d7e06f148524e3f0bdae2afb02e7819a0c24d1
Git SHA-1:                                     fe51f1ef5cfbcb4d5f559af39080963cf28ff1af
Configuration directory:                       /home/ubuntu/.mina-config
Peers:                                         54
User_commands sent:                            0
SNARK worker:                                  None
SNARK work fee:                                100000000
Sync status:                                   Synced
Catchup status:                                
    To build breadcrumb:           0
    To initial validate:           0
    Finished:                      304
    To verify:                     0
    To download:                   0
    Waiting for parent to finish:  0

Block producers running:                       1 (B62qrcQDeAfg2qSnZhdF8wQ78G9QmKNDqpESVt2SL43F97kF3Cyv32a)
Coinbase receiver:                             Block producer
Best tip consensus time:                       epoch=15, slot=1046
Best tip global slot (across all hard-forks):  108146
Next block will be produced in:                in 13.674h for slot: 108420 slot-since-genesis: 108420 (Generated from consensus at slot: 108064 slot-since-genesis: 108064)
Consensus time now:                            epoch=15, slot=1046
Consensus mechanism:                           proof_of_stake
Consensus configuration:                       
    Delta:                     0
    k:                         290
    Slots per epoch:           7140
    Slot duration:             3m
    Epoch duration:            14d21h
    Chain start timestamp:     2021-03-17 00:00:00.000000Z
    Acceptable network delay:  3m

Addresses and ports:                           
    External IP:    3.84.79.225
    Bind IP:        0.0.0.0
    Libp2p PeerID:  12D3KooWQBiq7tzFZ2uScLDrXvsk2ecTzWp3Kj7pFvQ5YKLXxPhT
    Libp2p port:    8302
    Client port:    8301

Additional information

No response

shimkiv commented 2 years ago

@nholland94 can you please take a look when time allows.