Consensys / defi-score

DeFi Score: An open framework for evaluating DeFi protocols
https://defiscore.io
Other
279 stars 79 forks source link

Adding centralization risk sub-component #17

Closed jclancy93 closed 4 years ago

jclancy93 commented 4 years ago

Problem Statement

Centralization risk is an important risk to consider when lending money with DeFi protocols. Different DeFi protocols have different levels of centralization risk, as the DeFi Score should attempt to highlight them. The requirements for this solution should accomplish several things:

Protocol Administration

One of the biggest contributors to centralization risk in DeFi protocols is the use of admin keys. Admin keys allow protocol developers to change different parameters of their smart contract systems like oracles, interest rates and potentially more. Protocol developer’s’ ability to alter these contract parameters allows them to cause financial loss to users. Measures like timelocks and multi-signature wallets help mitigate the risk of financial loss due to centralized elements. Mult-signature wallets help mitigate this risk by distributing control to a larger number of developers, meaning that the loss or compromise of a single private key cannot compromise the entire system. Timelocks help mitigate risk by allowing protocol users to exit their positions before a change can take place.

Score Description
1 Admin keys without timelock
2 Admin keys with timelock
3 Admin keys with timelock and multisig
4 No admin keys - autonomous/decentralized governance

Oracles

Another large element of centralization risk in these protocols is oracle centralization. There are many different flavors of oracle systems being used to power these protocols. Some protocols use a fully self-operated oracle system while others use externally operated oracles like Uniswap and Kyber. Samczun’s writeup on oracles and their ability to cause financial loss provides good background information. The oracle centralization score is not focused on whether these price feeds are manipulatable or not (they all are), but whether a single entity can manipulate them with ease. In the self-operated model, it only takes the oracle owner to manipulate its data. Decentralized oracles can’t be manipulated in the same way, but may not always represent the fair market value for an asset, which is why developers building on top of decentralized oracles opt to use price volatility bounds to defend against these types of attacks.

Score Description
1 Only centralized self-operated oracles used
2 Mix of centralized and decentralized oracles used
3 Only decentralized oracles used
4 No oracles needed

Results

Protocol Score
Uniswap 8/8
Compound 5/8
dYdX 5/8
Nuo 3/8
Fulcrum 5/8
Maker 7/8

Uniswap The constant product AMM model means that Uniswap doesn’t require a price feed, it is a price discovery mechanism itself. It scores the highest possible (4/4) on oracle centralization. Also, Uniswap is a fully autonomous set of smart contracts, so it gets the highest possible score on protocol administration (4/4)

Compound While Compound recently announced a new, more decentralized oracle system, it has still not been integrated into the Compound protocol. Their current oracles are solely managed and operated by them, resulting in a score of (2/4). Compound also recently released an upgrade to its governance model that uses a timelock for protocol administration of >2days and uses an off-chain multisig system. Compound scores (3/4) on protocol administration.

dYdX dYdX uses a mix of centralized and decentralized oracles. They use Maker’s oracles, which for the scope of this framework can be considered decentralized because it relies on a distributed network of reporters. For Dai, they use a centrally administered contract that pulls from Eth2Dai, Maker’s oracle, and Uniswap on request. Dydx uses a mix of centralized and decentralized oracles so they receive a (2/4) on oracle centralization. Dydx’s admin controls use both a multisig and a timelock, so they receive (3/4) on protocol administration

Alternatives

There were many other variables that could have been selected in addition to/in replace of the protocol administration and oracles. While attributes like emergency shutdowns, upgradable contracts, IPFS hosted frontends, etc. are important, many of these controls are dependent on the model of protocol administration. To keep the framework as simple as possible, the variables that introduce the largest avenues for centralization to exist in a protocol were included.

Proposed Weights

Component Weight
Smart Contract Risk 45%
Financial Risk 30%
Centralization Risk 25%
sneg55 commented 4 years ago

How's Uniswap related to DeFi score framework "for assessing risk in permissionless lending platforms." ?

jclancy93 commented 4 years ago

It's not. Sorry, should have included a note there. Even though they are currently covered by DFS, I added Uniswap and Maker to illustrate the full spectrum of scores that DeFi platforms currently achieve.

sneg55 commented 4 years ago

I think it'll confuse people, especially those who not really familiar with the DeFi space.

brockelmore commented 4 years ago

From a liquidity providing perspective, I think it makes sense to include them, imo.

sneg55 commented 4 years ago

Comparing apples to oranges? what's the point?

EthWarrior commented 4 years ago

Some weight could be added to the the oracle network state, i.e. how many nodes are within the oracle network and what kind of financial value it takes to break the network.

EthWarrior commented 4 years ago

Also the listed assets itself has centralization risk as well to some extent (for example USDC has a custody risk).

jclancy93 commented 4 years ago

@sneg55 @brockelmore the thought behind including it was just to provide a full spectrum of scores. These results won't be going into the whitepaper or anything, just for illustrative purposes.

@EthWarrior I like the concept of collusion resistant oracles. I think some of the work UMA has been doing on that front is extremely interesting. I would be interested to hear more in-depth thoughts you have on measuring the financial cost of collusion.

The DeFi Score doesn't currently consider the assets defi markets are denominated in, but it's definitely something worth considering and could be a great addition to the score in the future.

Da-Crypto commented 4 years ago

Looks interesting! How many ratio do you plan to put this matter? (For example currently 50% ratio is put in "Smart contract risk".)

jclancy93 commented 4 years ago

@Da-Crypto I've included a new section in the PR description that outlines the proposed weights.

We are also proposing the removal regulatory risk for the time being. We do not have legal expertise on our team and therefore find it difficult to give an accurate assessment of regulatory risk

Da-Crypto commented 4 years ago

Thank you! Regularity lisk looks important too.

Could you tell us the reason to put this ratio? It makes beter understanding for community. I think it ok if which is just temporally ratio.

jclancy93 commented 4 years ago

@Da-Crypto agreed that regulatory risk is definitely important. I think there's quite a bit of overlap between centralization risk and regulatory risk. It is quite difficult to regulate/censor something that is completely decentralized.

Centralization risk has been giving a larger ratio than regulatory risk because with a centralized platform you not only need to worry if it will be shut down by the government, but also that the operators of the platform will act in good faith

themandalore commented 4 years ago

Great push overall.
But can Maker's oracle really be considered decentralized? DPOS or known validators (POA) would hardly fly as fully decentralized for base layer chains, so I don't know why it would be any different for oracles

jclancy93 commented 4 years ago

@themandalore Thanks! Agreed that it's not fully decentralized. However, there are very few examples of fully decentralized oracles being used in production today. Hopefully this will be changing in the future as projects like Trellor gain adoption 😃

Maybe distributed is a more apt term than decentralized. What do you think?

themandalore commented 4 years ago

@jclancy93 , definitely! I just think there's a big range between Maker's oracle and 'no oracle needed' in terms of decentralization. In fact, I'd argue for more of a system:

Score | Description
-- | --
1 | Only centralized self-operated oracles used
2 | Multiple and/or distributed oracles used
3 | Only decentralized oracles used
4 | No oracles needed