search

crypto-org-chain / cronos

Cronos is the first Ethereum-compatible blockchain network built on Cosmos SDK technology. Cronos aims to massively scale the DeFi, GameFi, and overall Web3 user community by providing builders with the ability to instantly port apps and crypto assets from other chains while benefiting from low transaction fees, high throughput, and fast finality.
Other
292 stars 234 forks source link

Problem: state sync gets stuck #944

Closed mr-kenikh closed 1 year ago

mr-kenikh commented 1 year ago

Describe the bug I can't synchronize the node using statesync, every time the synchronization gets stuck on the last 1-2 chunks like this:

8:14PM INF Applied snapshot chunk to ABCI app chunk=258 format=2 height=7620500 module=statesync server=node total=260

To Reproduce https://docs.cronos.org/for-node-hosts/running-nodes/cronos-mainnet/state-sync

Additional context I tried several times today and several weeks ago, with different internet connections (1 Gbps, 2 Gbps and 10Gbps), I used addrbook.json from an already running node, I used different versions of the binary (1.0.2 and 1.0.5), but both using rocksdb – and all times it gets stuck.

mmsqe commented 1 year ago

@mr-kenikh I try statesync locally without issue, do you know the memory usage? Might need at least 32 GB.

mr-kenikh commented 1 year ago

@mmsqe yes, I am using 64 GB, right now I am trying to sync with leveldb and I faced the same problem:

10:59AM INF Applied snapshot chunk to ABCI app chunk=256 format=2 height=7630000 module=statesync server=node total=260
10:59AM INF Applied snapshot chunk to ABCI app chunk=257 format=2 height=7630000 module=statesync server=node total=260
10:59AM INF Applied snapshot chunk to ABCI app chunk=258 format=2 height=7630000 module=statesync server=node total=260

No state sync error messages appeared after that, I will leave the process running, maybe get the synchronization to complete, however it is about out of memory:

MiB Mem :  64306.2 total,    351.3 free,  63785.0 used,    170.0 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.    348.5 avail Mem
mr-kenikh commented 1 year ago

It looks like state sync didn't get stuck because process was finally killed due out of memory:

Killed

So just need more time and memory. Now I will try to synchronize with 96 GB.

mr-kenikh commented 1 year ago

@mmsqe

12:48PM INF Fetching snapshot chunk chunk=258 format=2 height=7630000 module=statesync server=node total=260
12:48PM INF Applied snapshot chunk to ABCI app chunk=257 format=2 height=7630000 module=statesync server=node total=260
12:48PM INF Fetching snapshot chunk chunk=259 format=2 height=7630000 module=statesync server=node total=260
12:48PM INF Applied snapshot chunk to ABCI app chunk=258 format=2 height=7630000 module=statesync server=node total=260

An hour later the 259 chunk was applied:

1:50PM INF Applied snapshot chunk to ABCI app chunk=259 format=2 height=7630000 module=statesync server=node total=260

And finally process crashed again:

1:51PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 12312 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xcb777c9f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc76abce9f0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc9e47028c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc9e47028c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc9e47028c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc9e47028c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc9e47028c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc9e44dc800, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc9e44dc800?, 0x0?, 0x2b80?}, 0xae?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc9e44dc800?, 0xe1e160?, 0xc0013d5de0?}, 0x777cdb50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc9e44dc800?, 0xc9d853a2e8?, 0x12?}, 0xd853a300?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc9e44dc800, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc9cdf4fc80?, 0xcb777cdcd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc9d7f1cd00}, {0x38c51b0, 0xc9e0743c20}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc9d855f000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xcb5bad18c0)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1f7176c]

goroutine 11663 [running]:
github.com/evmos/ethermint/x/feemarket/keeper.Keeper.CalculateBaseFee({{0x38da4f0, 0xc0018dc230}, {0x38acf00, 0xc001c4fa50}, {0x38acf50, 0xc001c4fa90}, {{0x38da4f0, 0xc0018dc230}, 0xc0004a8008, {0x38acf00, ...}, ...}}, ...)
    github.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/x/feemarket/keeper/eip1559.go:47 +0x60c
github.com/evmos/ethermint/x/feemarket/keeper.(*Keeper).BeginBlock(_, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc9e47d4480}, {{0xb, 0x0}, {0xc6a08cb770, 0x12}, 0x746cb1, ...}, ...}, ...)
    github.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/x/feemarket/keeper/abci.go:15 +0xdb
github.com/evmos/ethermint/x/feemarket.AppModule.BeginBlock(...)
    github.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/x/feemarket/module.go:148
github.com/cosmos/cosmos-sdk/types/module.(*Manager).BeginBlock(_, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc9e47d4480}, {{0xb, 0x0}, {0xc6a08cb770, 0x12}, 0x746cb1, ...}, ...}, ...)
    github.com/cosmos/cosmos-sdk@v0.46.11/types/module/module.go:484 +0x1bb
github.com/crypto-org-chain/cronos/app.(*App).BeginBlocker(...)
    github.com/crypto-org-chain/cronos/app/app.go:819
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).BeginBlock(_, {{0xc9e40d6ee0, 0x20, 0x20}, {{0xb, 0x0}, {0xc6a08cb770, 0x12}, 0x746cb1, {0x188f41dd, ...}, ...}, ...})
    github.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:183 +0x843
github.com/tendermint/tendermint/abci/client.(*localClient).BeginBlockSync(_, {{0xc9e40d6ee0, 0x20, 0x20}, {{0xb, 0x0}, {0xc6a08cb770, 0x12}, 0x746cb1, {0x188f41dd, ...}, ...}, ...})
    github.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:280 +0x118
github.com/tendermint/tendermint/proxy.(*appConnConsensus).BeginBlockSync(_, {{0xc9e40d6ee0, 0x20, 0x20}, {{0xb, 0x0}, {0xc6a08cb770, 0x12}, 0x746cb1, {0x188f41dd, ...}, ...}, ...})
    github.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:81 +0x55
github.com/tendermint/tendermint/state.execBlockOnProxyApp({0x38c9968?, 0xc001c790e0}, {0x38d2e50, 0xc001cb4ab0}, 0xcb4d530d20, {0x38dbd88, 0xc00061a798}, 0x746cb0?)
    github.com/tendermint/tendermint@v0.34.27/state/execution.go:307 +0x51d
github.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock(_, {{{0xb, 0x0}, {0x288c516, 0x7}}, {0xc001c29620, 0x12}, 0x1, 0x746cb0, {{0xc001670d40, ...}, ...}, ...}, ...)
    github.com/tendermint/tendermint@v0.34.27/state/execution.go:140 +0x171
github.com/tendermint/tendermint/blockchain/v0.(*BlockchainReactor).poolRoutine(0xc00107f6c0, 0x1)
    github.com/tendermint/tendermint@v0.34.27/blockchain/v0/reactor.go:400 +0xbda
created by github.com/tendermint/tendermint/blockchain/v0.(*BlockchainReactor).SwitchToFastSync
    github.com/tendermint/tendermint@v0.34.27/blockchain/v0/reactor.go:127 +0xe5

Btw, during state syncing there are a lot of ERR ante handler panicked error="invalid denom: " like this:

12:26PM INF Applied snapshot chunk to ABCI app chunk=6 format=2 height=7630000 module=statesync server=node total=260
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc00227b320})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356df700}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356df700}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356df700}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356df700}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356df700}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc001edc000, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc001edc000?, 0xc0356e78f0?, 0x6c456e?}, 0x356e79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc001edc000?, 0xe1e160?, 0xc0013d5de0?}, 0x356e7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc001edc000?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc001edc000, 0x1ec, 0x1ec}, 0x430795?, {0xa178?, {0xc01fe69da0?, 0x23951ec?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc0024125b8}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc00227b5f0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356dfec0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356dfec0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356dfec0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356dfec0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0356dfec0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc001edd200, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc001edd200?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc001edd200?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc001edd200?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc001edd200, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc002412ba0}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc00227b8f0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675e680}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675e680}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675e680}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675e680}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675e680}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc001edd800, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc001edd800?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc001edd800?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc001edd800?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc001edd800, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cc8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc002412d38}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc00227bbc0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675ee40}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675ee40}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675ee40}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675ee40}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675ee40}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc0005b0400, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc0005b0400?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc0005b0400?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc0005b0400?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc0005b0400, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc002412f00}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc00227be90})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675f600}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675f600}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675f600}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675f600}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675f600}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc0005b0a00, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc0005b0a00?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc0005b0a00?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc0005b0a00?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc0005b0a00, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc002413098}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c38160})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675fdc0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675fdc0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675fdc0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675fdc0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03675fdc0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc0005b1200, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc0005b1200?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc0005b1200?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc0005b1200?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc0005b1200, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc002413230}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c38430})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679a580}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679a580}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679a580}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679a580}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679a580}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc0005b1e00, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc0005b1e00?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc0005b1e00?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc0005b1e00?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc0005b1e00, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc0024133c8}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c38700})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679ad40}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679ad40}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679ad40}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679ad40}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679ad40}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc000180a00, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc000180a00?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc000180a00?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc000180a00?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc000180a00, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc002413560}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c389d0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679b500}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679b500}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679b500}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679b500}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679b500}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc000181000, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc000181000?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc000181000?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc000181000?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc000181000, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc0024136f8}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c38cb0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679bcc0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679bcc0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679bcc0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679bcc0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc03679bcc0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc000181800, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc000181800?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc000181800?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc000181800?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc000181800, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc002413890}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc0017af580})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036c85040}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036c85040}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036c85040}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036c85040}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036c85040}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc001cb9000, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc001cb9000?, 0xc001cab8f0?, 0x6c456e?}, 0x1cab9e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc001cb9000?, 0xe1e160?, 0xc0013d5de0?}, 0x1cabb50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc001cb9000?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc001cb9000, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc001cabcd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc00002cdb0}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c39e60})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18000}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18000}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18000}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18000}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18000}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc001437800, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc001437800?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc001437800?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc001437800?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc001437800, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc002413a28}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001694130})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d187c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d187c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d187c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d187c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d187c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc000e47c00, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc000e47c00?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc000e47c00?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc000e47c00?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc000e47c00, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc002413bc0}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001694410})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18f80}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18f80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18f80}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18f80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d18f80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc00272d400, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc00272d400?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc00272d400?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc00272d400?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc00272d400, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc002413d58}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc0016946e0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19740}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19740}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19740}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19740}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19740}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc00272da00, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc00272da00?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc00272da00?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc00272da00?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc00272da00, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc002413ef0}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc0016949b0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19f00}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19f00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19f00}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19f00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d19f00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc001404800, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc001404800?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc001404800?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc001404800?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc001404800, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc001f6c090}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001694c80})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5e6c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5e6c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5e6c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5e6c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5e6c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc036d60000, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc036d60000?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc036d60000?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc036d60000?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc036d60000, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc001f6c270}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001694f50})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5ee80}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5ee80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5ee80}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5ee80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5ee80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc036d60600, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc036d60600?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc036d60600?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc036d60600?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc036d60600, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc001f6c450}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001695220})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5f640}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5f640}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5f640}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5f640}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5f640}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc036d60c00, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc036d60c00?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc036d60c00?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc036d60c00?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc036d60c00, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc001f6c5e8}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 430 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036cb3f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc0016954f0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5fe00}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5fe00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5fe00}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5fe00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036d5fe00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc036d61200, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc036d61200?, 0xc036cb78f0?, 0x6c456e?}, 0x36cb79e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc036d61200?, 0xe1e160?, 0xc0013d5de0?}, 0x36cb7b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc036d61200?, 0xc01cea30f0?, 0x10?}, 0x1ce85380?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc036d61200, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01cebe0c0?, 0xc036cb7cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001a4c000}, {0x38c51b0, 0xc001f6c780}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc0213e0000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc001f14420)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c4f300})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0325694c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0325694c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0325694c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0325694c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc0325694c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc001a67800, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc001a67800?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc001a67800?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc001a67800?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc001a67800, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc0021fb7d0}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c4f5e0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc032569c80}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc032569c80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc032569c80}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc032569c80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc032569c80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc001a67e00, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc001a67e00?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc001a67e00?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc001a67e00?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc001a67e00, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc0021fb9f8}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c4f8b0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2e440}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2e440}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2e440}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2e440}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2e440}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc036e24400, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc036e24400?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc036e24400?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc036e24400?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc036e24400, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc0021fbcf8}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c4fd10})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2ec00}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2ec00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2ec00}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2ec00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2ec00}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc036e24a00, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc036e24a00?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc036e24a00?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc036e24a00?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc036e24a00, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc0021fbe90}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc001c4ffe0})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2f3c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2f3c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2f3c0}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2f3c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2f3c0}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc036e25000, 0x1ec, 0x1ec})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc036e25000?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc036e25000?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc036e25000?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc036e25000, 0x1ec, 0x1ec}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc001c1c030}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f2, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
12:26PM ERR ante handler panicked error="invalid denom: " stack trace="goroutine 554 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/evmos/ethermint/app/ante.Recover({0x38c9968, 0xc00136b860}, 0xc036741f18)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:83 +0x11d\npanic({0x24bfb80, 0xc0018f4320})\n\truntime/panic.go:884 +0x213\ngithub.com/cosmos/cosmos-sdk/types.mustValidateDenom(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/coin.go:869\ngithub.com/cosmos/cosmos-sdk/types.DecCoins.AmountOf({0xc001a49bf0, 0x1, 0x1}, {0x0, 0x0})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/dec_coin.go:454 +0x511\ngithub.com/evmos/ethermint/app/ante.EthMempoolFeeDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2fb80}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:562 +0x485\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2fb80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.EthSetupContextDecorator.AnteHandle({{_, _}}, {{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2fb80}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/eth.go:524 +0x1d0\ngithub.com/cosmos/cosmos-sdk/types.ChainAnteDecorators.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2fb80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/types/handler.go:40 +0x136\ngithub.com/evmos/ethermint/app/ante.NewAnteHandler.func1({{0x38c83c0, 0xc0000540b8}, {0x38daf70, 0xc036e2fb80}, {{0x0, 0x0}, {0x0, 0x0}, 0x0, {0x0, ...}, ...}, ...}, ...)\n\tgithub.com/evmos/ethermint@v0.6.1-0.20221101220534-a8ea4eceb6d9/app/ante/ante.go:59 +0x416\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc00107f500, 0x0, {0xc036e25600, 0x1ef, 0x1ef})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/baseapp.go:685 +0x726\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).CheckTx(0xc00107f500, {{0xc036e25600?, 0xc0367458f0?, 0x6c456e?}, 0x367459e0?})\n\tgithub.com/cosmos/cosmos-sdk@v0.46.11/baseapp/abci.go:244 +0x86\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CheckTxAsync(0xc001c78b40, {{0xc036e25600?, 0xe1e160?, 0xc0013d5de0?}, 0x36745b50?})\n\tgithub.com/tendermint/tendermint@v0.34.27/abci/client/local_client.go:104 +0x118\ngithub.com/tendermint/tendermint/proxy.(*appConnMempool).CheckTxAsync(0xc0013d5de0?, {{0xc036e25600?, 0xc01fe63758?, 0x12?}, 0x1fe63770?})\n\tgithub.com/tendermint/tendermint@v0.34.27/proxy/app_conn.go:126 +0x26\ngithub.com/tendermint/tendermint/mempool/v0.(*CListMempool).CheckTx(0xc001c44300, {0xc036e25600, 0x1ef, 0x1ef}, 0xc001cb6b10?, {0x131e?, {0xc01fe69da0?, 0xc036745cd8?}})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/clist_mempool.go:254 +0x30d\ngithub.com/tendermint/tendermint/mempool/v0.(*Reactor).ReceiveEnvelope(0xc001c78fc0, {{0x38e9820, 0xc001325ee0}, {0x38c51b0, 0xc001c1c1f8}, 0x30})\n\tgithub.com/tendermint/tendermint@v0.34.27/mempool/v0/reactor.go:177 +0x4d8\ngithub.com/tendermint/tendermint/p2p.createMConnection.func1(0x30, {0xc013001000, 0x1f5, 0x1000})\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/peer.go:537 +0x445\ngithub.com/tendermint/tendermint/p2p/conn.(*MConnection).recvRoutine(0xc0018f8c60)\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:648 +0x430\ncreated by github.com/tendermint/tendermint/p2p/conn.(*MConnection).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.27/p2p/conn/connection.go:232 +0x238\n"
...

There was still enough memory left before the process crashed:

MiB Mem :  96566.4 total,    773.7 free,  73104.8 used,  22687.8 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.  22488.9 avail Mem
Screenshot 2023-03-30 at 14 55 56
JayT106 commented 1 year ago

@mr-kenikh I think it's the same issue discussed in https://github.com/crypto-org-chain/cronos/issues/317 and very much thanks for your effort.

There are two problems with it.

  1. CheckTx is causing panic when the node doing the state-sync from the snapshot.
  2. Certain snapshots on the network might broken, therefore the node is not able to proceed to the next block after the snapshot has been applied a broken snapshot in your node.

The current best way will be just using quicksync https://docs.cronos.org/for-node-hosts/running-nodes/cronos-mainnet/quicksync

And the CometBFT (the core consensus component from another team) will implement a local snapshot feature in Q2 so the node does not need to sync snapshots from the network instead of downloading the snapshot provided by us like the quicksync.

mr-kenikh commented 1 year ago

Today I was able to do state sync on mac arm:

3:12PM INF Fetching snapshot chunk chunk=260 format=2 height=7690000 module=statesync server=node total=262
3:12PM INF Applied snapshot chunk to ABCI app chunk=259 format=2 height=7690000 module=statesync server=node total=262
3:12PM INF Fetching snapshot chunk chunk=261 format=2 height=7690000 module=statesync server=node total=262
...
3:52PM INF Applied snapshot chunk to ABCI app chunk=260 format=2 height=7690000 module=statesync server=node total=262
...
3:53PM INF Applied snapshot chunk to ABCI app chunk=261 format=2 height=7690000 module=statesync server=node total=262
3:53PM INF Verified ABCI app appHash="!~l\x1d,j\x19-\x19\x02\x16mj" height=7690000 module=statesync server=node
3:53PM INF Snapshot restored format=2 hash="?\x1e_mX8\x16ӑdv\x1a\x16.h?" height=7690000 module=statesync server=node
3:53PM INF service start impl=BlockPool module=blockchain msg={} server=node
3:53PM INF burned tokens from module account amount=1346498829234066358178basecro from=evm module=x/bank
3:53PM INF minted coins from module account amount=1346498829234066358178basecro from=evm module=x/bank
3:53PM INF executed block height=7690001 module=state num_invalid_txs=0 num_valid_txs=2 server=node
3:53PM INF commit synced commit=436F6D6D697449447B5B31373920383820313037203531203234203237203132382031313720323036203338203737203132382032353020373120313239203232382032323720323337203638203133332037322032302032323320313232203232352031353420323132203233312035302030203934203139355D3A3735353731317D
3:53PM INF committed state app_hash=B3586B33181B8075CE264D80FA4781E4E3ED44854814DF7AE19AD4E732005EC3 height=7690001 module=state num_txs=2 server=node

And after that there was flooding:

3:53PM ERR failed to fetch block err="height 1 is not available, lowest height is 7690001" height=1 module=evmindex
3:53PM ERR failed to fetch block err="height 1 is not available, lowest height is 7690001" height=1 module=evmindex
3:53PM ERR failed to fetch block err="height 1 is not available, lowest height is 7690001" height=1 module=evmindex
...
3:53PM INF indexed block exents height=7690001 module=txindex server=node

I restarted the node and after that everything works fine.

mr-kenikh commented 1 year ago

I think this issue can be closed, but it should take attention to such errors as: ERR ante handler panicked error="invalid denom: " (btw, this error did not occur on the mac) ERR failed to fetch block err="height 1 is not available, lowest height is 7690001

yihuang commented 1 year ago

the log seems come from module=evmindex, we'll have a look.

yihuang commented 1 year ago

I just did some testings with state sync snapshot import/export, but didn't reproduce this issue, But it's indeed very slow, and there was even a memory leak (https://github.com/cosmos/iavl/pull/773). So hopefully with the memory leak fix, or even using the new memiavl feature, the state sync UX will improve a lot.

Closing for now, feel free to reopen if the issue persists.