FuelLabs / fuel-core

Rust full node implementation of the Fuel v2 protocol.
Other
58.15k stars 2.77k forks source link

Handle to big relayer log page size #2175

Open xgreenx opened 3 weeks ago

xgreenx commented 3 weeks ago

Overview

Some Ethereum RPC provides usually don't allow more than 10_000 events per response. Because of that, setting a big value for --relayer-log-page-size may lead to a stuck synchronization with L1.

We need to teach the Relayer service to adjust the page size.

Definition of done

When the RPC provider returns an error that the response is too big, we need to decrease the page size automatically.

If we decreased the page size, and for some period of time we didn't have any error, we should try to increase it.

The increased page size shouldn't be more than --relayer-log-page-size.

pwalski commented 2 weeks ago

I tried to set https://sepolia.infura.io/v3/.. RPC server as a relayer together with few servers from https://chainlist.org/chain/11155111. relayer-log-page-size was set to 1_000_000 in order to collect error responses.

Returned errors had no common messages nor a common error codes: code: -32701, message: "exceed maximum block range: 50000" code: -32005, message: "query returned more than 10000 results. Try with this block range [0x5AA536, 0x5BB3FF]." code: -32602, message: "query exceeds max block range 100000"

No idea how many different combinations of error codes and messages may indicate exceeding a block range limit. Even in case of infura code -32005 could indicate exceeding daily request count quota (https://docs.infura.io/api/networks/ethereum/how-to/avoid-rate-limiting) instead of exceeding a block range limit.

Disclaimer: I have no prior experience with eth RPC servers and I wanted to just implement some requested block range exponential backoff based on limit errors. I will not continue because the task requires a decision which cases of error responses should be handled and how to distinguish them from e.g. exceeding request count quota.