This is a budget request for maintaining all exporter-related tasks and adding new features if time permits. The exporter is a service that powers yearn.vision, calculates APYs for all our networks, serves data for those who would like to consume it, and is used to visualize current and historical vault, partner, treasury, and transaction data.
Highlights
The good
We built and integrated yETH into yearn.vision.
We have deployed Velodrome and Aerodrome apy previews to production so one can see yet to be deployed vault APYs for all these pools.
Aerodrome Previews APY
We have updated the Yearn-Exporter-API but more optimizations on the API are needed.
Added Base network and info on yearn.vision
Added Aerodrome forward-looking APY
Made substantial progress on Bal forward-looking APY
Added Speed and Async Optimizations to Exporter in various areas included the apy "s3" script:
fully implemented async code + dank on the main exporters, perf gains of 40-99.95% depending on exporter
new disk caches in both ypm and eth_portfolio enable data sharing between containers and reduce load on node, further reducing node dependency. More to come.
LogFilter class brings optimized log fetching / caching / decoding / reading / re-reading (big add tbh, for partners) as well as cleaner code
dank now supports tenderly
dank can now catch and handle 429 responses and now works with shitty public rpcs
Give the optimizations above it might be possible to now run exporter on less performant nodes
The bad
Archive nodes cost will be 300$ per node per network for unlimited usage. Previously it had been 100$ per node per network for a single node for the provider we were using. They had node downtime affecting our team and data availability to both the v1 api and yearn.vision. Also we will be dropping support for Arbitrum and Fantom due to inactivity and limited budget and adding Polygon network for v3 development.
Need to fix tests and add v3 support to exporter, we weren't able to finish this on the last budget request.
The current solution with victoria-metrics is stable and works but lacks flexibility for partial updates and is not easy to integrate into BI systems like metabase.
A new effort will be taken to create a datamodel with postgres-based timescale db which will enable us to have more control about the indexed web3 data and let us integrate it into 3rd party tools more easily. This will also help external data scientists to work with the exporter data more easily.
grafana dashboard statistics and enhancements
Building a statistics and machine learning dashboard in Grafana.
Expand dashboard concepts and tune ML dashboards to lower timeframes.
Build on PoC Dash and pull in realtime data for them, calculate statistics in docker in R.
Enhance and update dashboards to include customized graphs with statistics and metrics each.
fix tests
many older tests are failing which delays refactorings, leads to regressions and implies a lot of manual testing
migrate price-related tests to ypricemagic repo to reduce run time
yearn-exporter-api updates and improvements for partners and apps
work with teams to optimize pulling data from the exporter API and databases.
replace eth-brownie with ape once ape has async support
ape-dankmids
Deadline
2024-01-31
People
Crebsy and BobTheBuilder will take care of most of the maintenance, fixes, and if time permits, new feature additions to Exporter's codebase. 0xBasic is reviewing PRs, maintaining and improving grafana dashboards while CSdata1 works on integrating statistics and machine learning models into the Exporter. We will maintain and monitor the current APIs, the partner scripts, and data from Exporter.
Duration
This budget request is for a period of 3 months.
This budget request will let us run the exporter in "maintenance mode". This is a state where we dont add new features, unless time permits, and allows us to maintain the Exporter codebase and libraries with fixes and small improvements when needed. Thus lowering total costs while keeping everything working as normal: exporters, apis, monitoring.
Scope
This is a budget request for maintaining all exporter-related tasks and adding new features if time permits. The exporter is a service that powers yearn.vision, calculates APYs for all our networks, serves data for those who would like to consume it, and is used to visualize current and historical vault, partner, treasury, and transaction data.
Highlights
The good
We built and integrated yETH into yearn.vision. We have deployed Velodrome and Aerodrome apy previews to production so one can see yet to be deployed vault APYs for all these pools.
The bad
Archive nodes cost will be 300$ per node per network for unlimited usage. Previously it had been 100$ per node per network for a single node for the provider we were using. They had node downtime affecting our team and data availability to both the v1 api and yearn.vision. Also we will be dropping support for Arbitrum and Fantom due to inactivity and limited budget and adding Polygon network for v3 development.
Plan
Responsibilities
handle up status and monitoring for:
code maintenance for:
Tasks
finish Balancer apys
add yvaults v3 support
consolidate data layer for open data science:
grafana dashboard statistics and enhancements
fix tests
yearn-exporter-api updates and improvements for partners and apps
replace eth-brownie with ape once ape has async support
Deadline
2024-01-31
People
Crebsy and BobTheBuilder will take care of most of the maintenance, fixes, and if time permits, new feature additions to Exporter's codebase. 0xBasic is reviewing PRs, maintaining and improving grafana dashboards while CSdata1 works on integrating statistics and machine learning models into the Exporter. We will maintain and monitor the current APIs, the partner scripts, and data from Exporter.
Duration
This budget request is for a period of 3 months.
This budget request will let us run the exporter in "maintenance mode". This is a state where we dont add new features, unless time permits, and allows us to maintain the Exporter codebase and libraries with fixes and small improvements when needed. Thus lowering total costs while keeping everything working as normal: exporters, apis, monitoring.
Public Group
YFI Exporter - Telegram Group
Past budget request
yExporter #2
Money
Monthly breakdown: 41.67% cut for personnel
Amount
45570 DAI
Wallet address
0xcD63C69f08bdDa7Fe96a87A2Ca3f56f3a6990a75
Reporting
Once