Open wojciechos opened 2 months ago
Currently, Juno retries failed L1 requests immediately, causing potential spamming of l1 node. We need to introduce a delay between retries to manage load and improve system stability.
Sample logs - ~40 request in on sec
21:24:35.993 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.968 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.942 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.916 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.891 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.866 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.840 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.811 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.783 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.756 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.731 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.706 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.681 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.655 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.628 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.603 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.577 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.551 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.526 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.501 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.476 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.451 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.426 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.397 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.372 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.346 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.318 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.291 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.266 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.240 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.215 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.190 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.164 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.139 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.111 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.086 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.060 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.035 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"} 21:24:35.009 21/04/2024 +00:00 DEBUG l1/l1.go:180 Failed to retrieve L1 finalised height, retrying... {"error": "get finalised Ethereum block: daily request count exceeded, request rate limited"}
Currently, Juno retries failed L1 requests immediately, causing potential spamming of l1 node. We need to introduce a delay between retries to manage load and improve system stability.
Sample logs - ~40 request in on sec