search

ton-blockchain / TIPs

Improvement Proposal repository
78 stars 16 forks source link

Bug which caused network stop at June 31 #11

Closed EmelyanenkoK closed 3 years ago

EmelyanenkoK commented 4 years ago

Network has stopped at night June 30 - June 31. The following details were found in logs:

[ 1][t 1][1590883071.909589767][validate-query.cpp:81][!validateblock(-1,8000000000000000):481752]      REJECT: aborting validation of block candidate for (-1,8000000000000000) : parsing block header : cannot unpack block header : [Error : 0 : block header contains block id (-1,8000000000000000,481751), expected (-1,8000000000000000,481752)]
[ 1][t 1][1590883071.918973207][validator-session.cpp:338][!session]    [validatorsession GosDF/+iYWs7yzIELJRJLkSUER5a9Cp0qiYET/Yei78=@zCHuLva++LTCDo8DAe7haIki312TQSaAQhd4g7x4rA8=]: failed candidate 5U1/z6RMKTwvZWZuERERfgV9b/TyEvO6xF2Vbl5cH4A=: bad candidate: parsing block header : cannot unpack block header : [Error : 0 : block header contains block id (-1,8000000000000000,481751), expected (-1,8000000000000000,481752)]

also many

[ 1][t 6][1590883071.954610348][validator-session-state.cpp:923][!session]      VALIDATOR SESSION: NODE EZ+e4C1EIZEapXHVuTLyYSiAKLi4KqUh9RwN/4Fa724= REJECTED CANDIDATE HorTGu1VR9OQumLCInrJGH46460WymwTihQExKw9Cug= WITH REASON bad candidate: parsing block header : cannot unpack block header : [Error : 0 : block header contains block id (-1,8000000000000000,481752), expected (-1,8000000000000000,481751)]
[ 1][t 6][1590883072.347645044][validator-session-state.cpp:923][!session]      VALIDATOR SESSION: NODE AbUy93b49KVi73mdPjn20JRb4dOa4SDXeRlnJcTf/fw= REJECTED CANDIDATE 5U1/z6RMKTwvZWZuERERfgV9b/TyEvO6xF2Vbl5cH4A= WITH REASON bad candidate: parsing block header : cannot unpack block header : [Error : 0 : block header contains block id (-1,8000000000000000,481751), expected (-1,8000000000000000,481752)]

There were also some related errors and warning in logs, which at this moment we believe are secondary to the main problem. In particular:

Looks like some validator generated wrong block, or may be out of turn block. However, since there are many validators now and all of them control less than 33% of voting power for bug to stop network it is necessary that some other validators also accepted this block, while majority of others did not.