blockscout doesn't show blocks with optimism L2 transactions

[blockchaindevsh]

Description

Here's the log from op-geth(v1.101315.0) which shows transaction 0x187712a3e229498E9E42888761Ab9B92bceB46c7 is included into block 288903:

But the explorer(installed from production-optimis branch) shows the transaction is pending, and can't show the block 288903 here.

It can show the previous and next block(which are essentially empty) normally.

Is it because blockscout currently only supports old version of optimism?

Type of the installation

Docker-compose

Type of the JSON RPC archive node

Geth

Type of the chain

No response

Link to the page

http://142.132.154.16/

Steps to reproduce

No response

Backend version

3ed126d7484948887e62cd89ec55860a95c425043ed126d7484948887e62cd89ec55860a95c42504

Frontend version

3ed126d7484948887e62cd89ec55860a95c42504

Elixir & Erlang/OTP versions

N/A

Operating system

linux

Additional information

No response

[varasev]

Hi @blockchaindevsh, Is it reproducible at the moment? If so, can you please update the links above?

[blockchaindevsh]

@varasev yeah I found the root cause of the problem is that: when a l2 batch is timed out, l2 will reorg, which will rollback a huge number of blocks, like thousands of l2 blocks.

(This can happen if the batch submitting interval is long, like 12~24h)

Is there any consideration for this case?

[varasev]

So, after the reorg, explorer didn't show the reorged block but still shown its transaction as pending. Before the reorg, the same block and transaction were displayed normally. Everything is right?

[blockchaindevsh]

@varasev The problem is that the explorer doesn't reindex after the reorg, so the balances of accounts are stale.

I guess the root reason is that the reorg depth exceeds the finality depth assumed by explorer?

[varasev]

Did you try with smaller reorg depth? Does the issue still persist in this case?

[blockchaindevsh]

@varasev I didn't try smaller reorg depth. I think there's actually only 1 problem here, which is caused by : the reorg depth exceeds the finality depth assumed by explorer. Is the finality depth hardcoded?

[varasev]

@blockchaindevsh Can you please try to reproduce the issue and play with INDEXER_REALTIME_FETCHER_MAX_GAP env variable?

[blockchaindevsh]

@varasev What's the default value of INDEXER_REALTIME_FETCHER_MAX_GAP?

[varasev]

It's 1000

[blockchaindevsh]

@varasev I've two question about INDEXER_REALTIME_FETCHER_MAX_GAP:

1. Is INDEXER_REALTIME_FETCHER_MAX_GAP the maximum reorg depth allowed for blockscout?
2. Can INDEXER_REALTIME_FETCHER_MAX_GAP be adjusted for a running blockscout instance?

[varasev]

@blockchaindevsh

1. Yes for the realtime indexer. You can see its logic in this piece of code: https://github.com/blockscout/blockscout/blob/05d8d198d44a5735a13d2f0a584c5cc6c0e218ed/apps/indexer/lib/indexer/block/realtime/fetcher.ex#L133-L142

2. Yes, you can change its value and restart your instance.

[varasev]

But in the case when INDEXER_REALTIME_FETCHER_MAX_GAP is too small, the missed gap should be processed by catchup indexer. @Qwerty5Uiop can you please advise here why the catchup can skip the block gap handling?

[Qwerty5Uiop]

Hi @blockchaindevsh!

Is INDEXER_REALTIME_FETCHER_MAX_GAP the maximum reorg depth allowed for blockscout?

Actually, no, this variable was added just to handle cases when the latest block result may be too far from the real latest (because the nodes under the balancer are out of sync).

There is no maximum reorg depth allowed. Let's consider the situation when reorg happens on N-2000 height with INDEXER_REALTIME_FETCHER_MAX_GAP=1000 (N is the current latest block number). In this case realtime fetcher would stuck on the block N because the max gap is too small to recognize it as a real reorg. So the realtime fetcher will stay at the block N until the latest result would fit in this max gap. When this happens (the latest block equals or above N-1000) the realtime fetcher will import the N-1000 block. During this import, the consensus of N-1001 block will be set to false since its hash is not equal to N-1000 parent_hash. Then catchup fetcher will process the block N-1001 and during this import, the consensus of the N-1002 block will be set to false. And so on until the N-2000 where reorg happened. So eventually the chain will be relevant regardless of the INDEXER_REALTIME_FETCHER_MAX_GAP value.

So to speed up the process you should set the INDEXER_REALTIME_FETCHER_MAX_GAP to some reasonable value if 1000 is not enough. Or if there is no reasonable value and reorg can happen on any block, you can set it to some enormous value (for example the current latest block * 100).