timeout packet verification failed

[hallazzang]

I keep getting this error when I run the relayer:

2024-06-13T14:09:48.114538Z     error   Error sending messages  {"path_name": "...", "src_chain_id": "...", "dst_chain_id": "...", "src_client_id": "07-tendermint-...", "dst_client_id": "07-tendermint-...", "error": "rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 1: timeout packet verification failed: failed next sequence receive verification for client (07-tendermint-...): chained membership proof failed to verify membership of value: 0000000000000002 in subroot A851...D7E7 at index 0. Please ensure the path and value are both correct.: invalid proof [cosmos/ibc-go/v8@v8.2.1/modules/core/23-commitment/types/merkle.go:218] with gas used: '625040': unknown request"}

Is there any known reports about it? Here's my configuration file:

global:
    api-listen-addr: :5183
    timeout: 10s
    memo: ""
    light-cache-size: 20
    log-level: info
    ics20-memo-limit: 0
    max-receiver-size: 150
chains:
    chain1:
        type: cosmos
        value:
            key-directory: /root/.relayer/keys/...
            key: relayer
            chain-id: chain-1
            rpc-addr: http://...:26657
            account-prefix: ...
            keyring-backend: test
            gas-adjustment: 1.5
            gas-prices: 0.15...
            debug: false
            timeout: 20s
            output-format: json
            sign-mode: direct
            broadcast-mode: batch
    chain2:
        type: cosmos
        value:
            key-directory: /root/.relayer/keys/...
            key: relayer
            chain-id: chain-2
            rpc-addr: http://...:26657
            account-prefix: ...
            keyring-backend: test
            gas-adjustment: 1.5
            gas-prices: 0.15...
            debug: false
            timeout: 20s
            output-format: json
            sign-mode: direct
            broadcast-mode: batch
paths:
    path1:
        src:
            chain-id: chain-1
            client-id: 07-tendermint-...
            connection-id: connection-...
        dst:
            chain-id: chain-2
            client-id: 07-tendermint-...
            connection-id: connection-...
        src-channel-filter:
            rule: ""
            channel-list: []

I just substituted some values with ... to not disclose some information. I want to make sure if it's the relayer's issue or the chains' issue.

[duonghb53]

@hallazzang do you have check 2 chains running?

[hallazzang]

@hallazzang do you have check 2 chains running?

Sure the chains are indeed running.

[duonghb53]

2 chains run on local or docker. I see you use same port 26657.

[duonghb53]

And can you describe all step run relayer?

[Reecepbcups]

This may be an IBC-Go thing? Looks like the hermes relayer had this same issue in the past come up here: https://github.com/informalsystems/hermes/issues/697

Need more info for what networks you are doing this on. I suspect its one of the chains issue vs relayer due to the invalid proof (improper packet?)