search

sidestream-tech / unified-auctions-ui

Unified MakerDAO auctions
https://unified-auctions.makerdao.com
GNU Affero General Public License v3.0
16 stars 13 forks source link

Investigate fetching of Maker protocol parameters #15

Open valiafetisov opened 3 years ago

valiafetisov commented 3 years ago

Goal

Get Maker protocol parameters from the appropriate blockchain

Context

We should get the parameters of the Maker protocol (per collateral type or in their terminology, per ilk) in order to, for example, display information about when is the next price drop. You can see they are being fetched on https://daiauctions.com and you can investigate how it's done in the source code of this website here https://github.com/martinlsanchez/daiauctions (although the source code seems to be minified 🙁)

Also, current https://liquidations.makerdao.com portal (source can be found at https://github.com/makerdao/liquidations-portal/) also fetching those values as they display numbers such as dust limit. Most probably the best place to learn about that is to start by reading source code of the liquidations portal or via dai.js documentation (although dai.js docs are incomplete, does not include liquidations plugin).

Tasks

mmbaldev commented 3 years ago

Would you please explain what are the parameters that we need to fetch? (providing some examples would be nice)

florianstoecker commented 3 years ago

Of Course! So for https://daiauctions.com/ you can see that there are a lot of parameters (with unusual names) on top of the page: Dog Dirt, Dog Hole, Chop, Dirt, Hole, Buf, Step, Cut. Those values are mainly predefined and hard-coded into the chain. For example, the parameter "Cut"(1%) is the percentage an auction price is decreased by every "Step" (90 seconds in this case)

So for the first step, it would be awesome if you can investigate how they do it on https://daiauctions.com/ by looking into the code at https://github.com/martinlsanchez/daiauctions.

Does this answer your question? Otherwise just hit me up for a quick call :)

valiafetisov commented 3 years ago

Parameters like STEP, CUT and others presented in the top row on the https://daiauctions.com

This issue related to the https://github.com/sidestream-tech/auction-ui/issues/61, where we going to use those parameters throughout the text

mmbaldev commented 3 years ago

Does this answer your question? Otherwise just hit me up for a quick call :)

Yes perfect 👍

mmbaldev commented 3 years ago

https://github.com/martinlsanchez/daiauctions/blob/898fa6efc06dc2df4d38caf022c0f93a9a513627/src/js/flap.js#L973

They are using web3 package to fetch the data, however, based on sidestream-tech/auction-ui#17 we are going to use the ethersjs package, so I'm going to check the logic for fetching maker protocol parameters, however, for implementation would be different.

mmbaldev commented 3 years ago

Maker Liquidation 2.0 Module Essentials:

There are three main contracts to communicate with Liquidation protocol: For getting the parameters we have to connect to the following contracts, which will be elaborated technical wise later.

  1. Abacus
  2. Clipper
  3. Dog ( the liquidation contract )

Some essential terms: ilk: Liquidation Penalty per collateral urn: the Vault to be liquidated kpr: the address where DAI incentives will be sent

The Medianizer Contract

The medianizer (oracle) is the smart contract which provides Makers trusted reference price.

Basic structure, How to start?

The mentioned source code is implemented by pure html, css and js. So for each page it copied most of the code. As I mentioned in the previous comment this code uses the web3 library and it is defined in the script tag of each html page. At first you can open clip_eth_a.html and there you can find out it is using the /js/clip_eth_a_min.js js file. Since it is minified, you have to unminify it with this website: https://unminify.com

JS Files Initialization

For interacting with smart contracts on the ethereum blockchain in web3 you should use this method: web3.eth.Contract()) It gets the contract_address and json interface of the respective smart contract then web3 will auto convert all calls into low level ABI calls over RPC for you.

There are 4 contracts defined in the code:

  1. medianizerContract => This is a connect to medianizer Contract
  2. dogContract => This is a connect to Dog Contract
  3. calcContract =>This is a connect to Abacus Contract
  4. clipContract => This is a connect to Clipper Contract

For connecting to each of these contracts there are two constants defined:

Execution and Main methods

There are two main functions in the js files:

  1. updateGlobals(): which retrieve the parameters and calculate some other params based on it and update the html UI WHICH IS THE TARGET FOR THIS ISSUE
  2. fetchAuctions(): As it's name is descriptive enough, it will fetch the auctions (we will not go through this method in this documentation)

How updateGlobals() method works?

constant ILK = "0x4554482d41000000000000000000000000000000000000000000000000000000",

These are the following params and their related contract: chop: dogContract.ilks(ILK).chop dirt: dogContract.Dirt() hole: dogContract.Hole() buf: clipContract.buf() kicks: clipContract.kicks() step: calcContract.step() cut: calcContract.cut() Price MED: medianizerContract.getPastEvents("LogMedianPrice")

Q: @valiafetisov There are some calculation after getting the parameters from the related contracts (I have provided some example of it below) which I don't understand it correctly, If you know what's the reason, please explain it :):

n = 100 * (parseInt(t.chop) / 1e18 - 1),
a = parseInt(t.dirt) / 1e45,
i = parseInt(t.hole) / 1e45;

Other Web3 methods that has been used in the code:

These web3 methods are used in the code for some parameter calculations, there are two points to consider about them:

  1. This source code is using the very old version of the web3, and some of these methods may be deprecated by now.
  2. We probably won't use these methods as we are going to use the ethers js library, however, we have to find the similar methods in the ethers js lib.
mmbaldev commented 3 years ago

Different Contracts On Different Networks Address: https://github.com/makerdao/dai.js/tree/316bd7769274f0a30153436cc935497e778b65f9/packages/dai-plugin-liquidations/contracts

LukSteib commented 3 years ago

@valiafetisov Based on the discussion we had yesterday with vulcanize guys there might be a possibility to get the parameters we need querying their graphQL endpoint. In order to steer direction of the research, information on that might be helpful to be documented here.

valiafetisov commented 3 years ago

there might be a possibility to get the parameters we need querying their graphQL endpoint

Yes, the dai.js library is already doing that under the hood, when certain plugins/methods are called. So this issue is to figure out which methods are those.

Also, current https://liquidations.makerdao.com portal (source can be found at https://github.com/makerdao/liquidations-portal/) also fetching those values as they display such things as dust limit.

To summarise: fetching those parameters can be done in several different ways: fetching directly from the blockchain without tooling, or using dai.js methods which can also do it directly (but abstracted) and via some external caching service (graphql) endpoint. In this issue we need to document possible solutions and choose the easiest way to get those numbers.

fegbert commented 3 years ago

Liquidation Portal

Dai.js instance: For everything dai.js related, they’re calling their getMaker function. This, on first call, uses the dai.js function create to configure the behaviour of the library. This includes adding plugins like the Liquidation Plugin which is the plugin developed for the liquidation portal, and the important one for us.

There is also an option to use vulcanize, which is set by a function getVulcanizeParam which is commented out and currently always returns false, which means vulcanize is never used because, as the comment states, vulcanize wasn't ready yet.

// TODO remove hardcoded 'false' when vulcanize is ready

Hole & Dirt

Hole & Dirt values can be fetched per ilk. This is done with the getHoleAndDirtForIlk function. The function returns the getHoleAndDirtForIlk() function from the dai.js liquidation plugin. (No documentation sadly), which returns an object with 3 values:

{
    hole: BigNumber,
    dirt: BigNumber,
    diff: BigNumber
}

Hole and Dirt are self explanatory, but there's a third value diff. Not sure right now what this is. Difference?

Dust Limit

The Dust limit is fetched as part of the Auction Object. This is done by with the fetchAuctions function. It can either be used to get all auctions, or all auctions for a specific ilk. This then calls the getAllClips function which returns the getAllClips function of the liquidation plugin. I haven’t been able to get a result other than [], but there is an example object in a comment:

{
  "saleId": "4990",
  "pos": "5",
  "tab": "48150187464057295135313238110939183779274217273",
  "lot": "196969000000000000",
  "usr": "0xdaaFAe93C0e2A0226043E88a70aCF5be9b671124",
  "tic": "1595930405",
  "top": "6120186359841348405000000000000000000000000000",
  "active": true,
  "created": "2020-07-28T04:00:05",
  "updated": "2020-07-28T04:00:05"
}

(This appears to be outdated though, since in the actual function other keys are being used)

The result then gets transformed with the transformAuction function. This, as the name already gives away, transforms the returned object. This might be done for readability, or for safety against changes (only have to change in one place instead of everytime this object is used).

valiafetisov commented 3 years ago

one thing that came up was vulcanize. From what he said this could help get more information and Lukas also mentioned it in a meeting, but I can't find anything about it, so Flo said I should talk to you. Do you know more about this / is this actually something that would help?

Yes, vulcanize is a database that we earlier referenced to as a graphql endpoint here: https://github.com/sidestream-tech/auction-ui/issues/62#issuecomment-919936876

The address of the endpoint is https://api.makerdao.com/graphql (it requires no authentication token despite the error shown when opening this url in a browser).

As described above, this endpoint is actually semi-used in the liquidations-plugin, see where they request dust limit: https://github.com/makerdao/dai.js/blob/dev/packages/dai-plugin-liquidations/src/LiquidationService.js#L91-L99

You can use a tool like https://github.com/OneGraph/graphiql-explorer to explore the endpoint yourself and check which data is available there. On the right side you will find an explorer where you can find all queries and their parameters.

Screenshot 2021-09-28 at 14 02 31

Example query will be (but there is more parameters available, as you can discover on the right):

{
    allIlks(first: 100) {
        nodes {
            id
            blockNumber
            rate
            art
            spot
            line
            dust
        }
        totalCount
    }
}

Recently I've also discovered that vulcanize source code is also available here https://github.com/makerdao/vdb-mcd-transformers

florianstoecker commented 3 years ago

Hole and Dirt are self explanatory, but there's a third value diff. Not sure right now what this is. Difference?

Let me clarify this question to complete your answer Hole is The Maximum amount of debt auctionable (at the same time). Dirt is the Current amount of debt being auctioned.

--> My assumption would be the same as yours: diff is the difference between Hole and Dirt, so in context the Left amount of debt auctionable (at the same time).

valiafetisov commented 3 years ago

In order to get actual price of an auction getStatus method should be called on the respective clipper contract, passing id of the auction. This will return:

dai.js liquidation plugin seems to have additional helper set up for this purpose:

await maker.service('liquidation').getStatus(collateralType, auctionId);
valiafetisov commented 3 years ago

Based on the problem that abacus contract (which does price drops) is actually quite complicated and have different decrease strategies (not only based on STEP/CUT), we decided to postpone this issue