search

forbole / callisto

Callisto (formerly BDJuno) is the official Juno implementation for Big Dipper, the open-source block explorer for Cosmos
https://bigdipper.live
MIT License
90 stars 241 forks source link

ERR re-enqueueing failed block: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg #701

Open RaulBernal opened 7 months ago

RaulBernal commented 7 months ago
## Bug description

I get this error in loop: 

11:07AM DBG processing block height=11134297
11:07AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=11134297
11:07AM DBG processing block height=11134297
11:07AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=12324351
image

As consequence, the MainNET explorer is 10 hours behind the current synced block.

image

I have another instance (a backup server) running the same software and is not stucked in the same ERR

image

Steps to reproduce

BDJuno version is (both servers): ``` bdjuno version version: 4.0.0-51-g340f59b4 commit: 340f59b4d0b3faeb00aa99bf69f17a5b95339dfb go: go1.21.6 linux/amd64 ``` ## Expected behavior

Expected a bypass if it can't parse the message not a stuck :)

RaulBernal commented 7 months ago

Maybe this issue could be related with this other https://github.com/cosmos/relayer/issues/1359#event-11647758685 and this race condition: https://github.com/cosmos/relayer/pull/1363 @RiccardoM

RaulBernal commented 7 months ago

I've asked related validators that please turn off the buggy relayer until I check if is related

RaulBernal commented 7 months ago

Maybe this issue could be related with this other cosmos/relayer#1359 (comment) and this race condition: cosmos/relayer#1363 @RiccardoM

Not related, is about Client Updates, not about IBC acks (so forget that comment)

MonikaCat commented 7 months ago

Hey @RaulBernal can I have a link to your bdjuno implementation? I think you can first try replacing juno version to the latest one v5.2.1-0.20240201075935-851426ddd905 and pulling any changes from BDJuno branch cosmos/v0.47.x Also you should add new unbonding_tokens column inside staking_pool table to avoid the staking pool error

RaulBernal commented 7 months ago

Hey @RaulBernal can I have a link to your bdjuno implementation? I think you can first try replacing juno version to the latest one v5.2.1-0.20240201075935-851426ddd905 and pulling any changes from BDJuno branch cosmos/v0.47.x Also you should add new unbonding_tokens column inside staking_pool table to avoid the staking pool error

Upgraded and running, but error still there: 10:01AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=12401971

How can I link you? (maybe pass me your SSH public key by DM using email/telegram/discord ? )

RaulBernal commented 7 months ago

For give you more context:

RaulBernal commented 7 months ago

So maybe I need to exec: bdjuno migrate v5 @MonikaCat

I have two identical servers; If I try to make the migration in one of them I get: bdjuno migrate v5 ERROR: error while migrating database: pq: relation "message_type" does not exist

RaulBernal commented 7 months ago

BTW added two collumns to staking_pool to fix one of the errors:

image
rider-cantuaria-eng commented 4 months ago

3:25PM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=14582098

Any idea guys?

running this config image

on callisto branch cosmos/v0.47.x branch

RaulBernal commented 2 months ago

Hey @RaulBernal can I have a link to your bdjuno implementation? I think you can first try replacing juno version to the latest one v5.2.1-0.20240201075935-851426ddd905 and pulling any changes from BDJuno branch cosmos/v0.47.x Also you should add new unbonding_tokens column inside staking_pool table to avoid the staking pool error

Certainly, error has disappears adding:

RaulBernal commented 2 months ago

Hi @MonikaCat I'm using BDjuno branch cosmos/v0.47.x with no changes. I started a new fresh installation using this release of juno v5.2.1-0.20240201075935-851426ddd905.

Everything started smooth but again:

10:31AM DBG processing block height=14729369
10:31AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=14729369
10:31AM DBG processing block height=14729369
10:31AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=14729369
10:31AM DBG processing block height=14729369
10:31AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=14729369
10:31AM DBG processing block height=14729369
10:31AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=14729369
10:31AM DBG processing block height=14729369
10:31AM DBG processing block height=14729369
10:31AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=14729369
10:31AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=14729369
10:31AM DBG processing block height=14729369
10:31AM DBG processing block height=14729574
10:31AM ERR re-enqueueing failed block err="failed to get transactions for block: error while unpacking message: no concrete type registered for type URL /ibc.core.client.v1.MsgUpdateClient against interface *types.Msg" height=14729574

It stucks the live sync (https://explorer2.bitcanna.io)

When I check in other explorer the block 14729369: (https://www.mintscan.io/bitcanna/block/14729369?chainId=bitcanna-1) I get some IBC client updates & IBC ACKs