search

umee-network / umee

A Golang implementation of the Umee network, a decentralized universal capital facility in the Cosmos ecosystem.
Apache License 2.0
225 stars 169 forks source link

Price feeder limitations #792

Closed adamewozniak closed 1 year ago

adamewozniak commented 2 years ago

Summary

Our price feeder has a few limitations that will affect our ability to support coins in the long term.

Problem Definition

Our price feeder is limited by :

Currently for the price feeder to vote on an exchange rate, it has to have at least three providers. This is to avoid scenarios where one provider goes down and we become dependent on another, which might malfunction or have low enough volume to be manipulated. This creates a pretty big issue in the cosmos ecosystem, because there are quite a few tokens only on one or two exchanges, such as osmosis and juno.

So, the right number of providers to require for an exchange rate is the number of providers which list that rate.

Certain APIs have stale data, such as the osmosis API we're using, which consistently returns historical data from ~6-4 minutes ago. The price feeder only uses the most recent information, so it filters out any candles that occurred before a certain amount of time, currently set to three minutes. So, this rule keeps us further from supporting coins that are only on osmosis.

So really, the right number of providers to require for an exchange rate is the number of providers which list that rate, and has an up-to-date, low-latency API available.

Proposal

We should :

  1. Use an external data source (or sources) like coingecko to determine the amount of exchanges on which a pair is listed, and require that amount instead of the default of "3". a. Need to decide on what to do in the case that these APIs fail; do we require 3 as the default or are we more lenient to support those currencies which only have one or two exchange rates?

  2. Find a way to get more recent pricing data from osmosis. This would probably require us to run our own osmosis node & API instead of using imperator's. a. Still need to look into endpoint limitations. If this data isn't accessible, how do we determine what is an acceptable time period per provider?

For Admin Use

adamewozniak commented 2 years ago

Looking into solving the osmosis issue first since it blocks the most. We should aggregate the "Swap Exact Amount In" Transactions (e.g. this ATOM/ROWAN tx) and then use this to build an OHCL websocket.

The spot endpoint doesn't have support for volume, so we'll need to query transactions for both of these

Check out this repo for updates on development of this API :^)

RafilxTenfen commented 2 years ago

About the proposal (1) We also could check on our own the number of exchanges on which a pair is listed using the providers GetAvailablePairs function ^^

adamewozniak commented 2 years ago

@RafilxTenfen yeah but this needs to also cover providers which we haven't implemented. For example, we need to be aware of this scenario:

  1. a coin gets listed only on osmosis
  2. we hard-code a rule that allows a single provider for that coin, release the price feeder, and then get it on chain
  3. the coin gets listed somewhere else

We'd have to then implement the new provider as soon as possible, especially if it has high volume

adamewozniak commented 1 year ago

Imo this can be solved for blue chip assets in the short-term by adding osmosis support, since mexc & bitget have both been added as providers for the price feeder

adamewozniak commented 1 year ago

Adding @rbajollari onto this since he's working on using coingecko for # 1 🥳

RafilxTenfen commented 1 year ago
  1. Use an external data source (or sources) like coingecko to determine the amount of exchanges on which a pair is listed, and require that amount instead of the default of "3".

Would be good to have at least 2 sources of exchanges... to avoid things like this again

adamewozniak commented 1 year ago
  1. Use an external data source (or sources) like coingecko to determine the amount of exchanges on which a pair is listed, and require that amount instead of the default of "3".

Would be good to have at least 2 sources of exchanges... to avoid things like this again

Agree, atm we mitigated this in #1496 by adding an override. That way the price feeders don't go down when coingecko goes down.

This can probably be closed after #1470 & we open a few tickets around cleanup like this ^

adamewozniak commented 1 year ago

Closing this out 🥳