dDemeter-fetch is a member of the demeter family, which is responsible for fetching and processing on-chain data. It provides standardized data for backtesting and defi research.
graph TD;
demeter-fetch([demeter-fetch]) --> demeter([demeter]);
demeter-fetch --> DeFi-research([DeFi research]);
Demeter-fetch can fetch chain event logs of DEFI projects from various sources and convert them into a unified format. In the project dimension, it supports Uniswap and AAVE. In the data source dimension, the supported data sources include:
Demeter-fetch supports export formats such as:
The minutely OHLC is mainly used in Demeter, while tick data is used in research, where being transformed into many forms.
Demeter-fetch uses streaming processing. It will summarize all the channels into raw files first, and then transform them into various forms. The processing flow is shown in the figure, in which green blocks are finished, and yellow blocks are to-do.
flowchart TB;
RPC:::done --rpc request--> raw:::done;
BigQuery:::done --api request--> raw;
Chifra:::done --cli export--> raw;
raw --OHLC--> minute:::done;
raw --formatted transaction--> tick:::done;
raw --> Squeeth:::done;
tick --> Uniswap;
tick --> AAVE;
tick --> other_defi;
subgraph Uniswap;
direction TB;
position:::done;
address_LP[LP of address]:::done;
fee_on_tick[fee on tick]:::to_do;
return_rate[return rate]:::to_do;
end
subgraph AAVE;
aave_tick[tick]:::done;
end
subgraph other_defi;
other_tick[tick]:::to_do;
end
classDef done fill:#00ff00
classDef to_do fill:#ffff00
chaoslab research: At this juncture, we retrieve all on-chain swaps for the selected pool over a span of 180 days, enabling us to replay them during the backtesting phase. This data retrieval is facilitated via Google BigQuery, which is employed by Demeter intrinsically. We extend our gratitude to the Demeter team for the creation and open-sourcing of this valuable library to the wider Uniswap community.
zelos research It is our own research. Demeter fetch have us won first prize in Bounty #20 of UGPCA