mento-protocol / reserve-site

Mento Reserve Website
https://reserve.mento.org
1 stars 3 forks source link

fix: Use RPC node to query for balances instead of apis #50

Closed bowd closed 7 months ago

bowd commented 7 months ago

Description

The reason we were having issues was that we were very often hitting the etherscan rate limit of 5 requests per second. This was also combined with a code-path that I left untreated, specifically this:

try {
    const response = await fetch(
      `https://api.etherscan.io/api?module=account&action=balance&address=${address}&tag=latest&apikey=${API_KEY}`
    )
    const data = (await response.json()) as EthScanBalanceResponse
    if (data.status === "0") {
      throw new Error("etherscan: status is zero")
    }
    return providerOk(formatNumber(data.result), Providers.etherscan)
  } catch (error) {
    return providerError(error, Providers.etherscan)
  }

The status === "0" check. At the time I didn't figure out that status zero means rate-limited and I left it there hoping that we would catch the issues in Sentry but never did 'cose we didn't wire Sentry into the backend. So basically even tho we were using either etherscan or ethplorer to query balances, when we hit etherscan's rate limit it killed the whole request. And this was what I caught in the front end as well.

Also, the reason why this started happening more is clearly because we've grown the number of queries we need to do to get all balances.

In order to mitigate this I replaced both querying of ether balance and ERC20 balance with RPC calls through Infura, removed the functions from Etherscan, and killed Ethplorer altogether. This is done through a batch json rpc provider so it's relatively fast. Using multicall would make it faster but would require way more refactoring that's not necessary.

Other changes

Wired up the Sentry exceptions to the API routes

Tested

Tested locally and diffed with existing data.

Related issues

Backwards compatibility

Yes

Documentation

N/A

vercel[bot] commented 7 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
reserve-site ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 30, 2023 11:43am