delvtech / agent0

Analysis & simulation repo for Delv
https://agent0.readthedocs.io/en/latest/
Apache License 2.0
52 stars 21 forks source link

Multi-pool and events based database refactor #1467

Closed slundqui closed 5 months ago

slundqui commented 5 months ago

This PR is the second in a collection of refactors to allow agent0 to support multiple pools.

This PR focuses on updates to the database and related systems. In particular, this makes the following major changes to the database.

Database schema updates

Analysis from remote chains

Interactive analysis interface

The interactive interface for getting information from the db has been reworked. Note that some pool analysis functions are only available in LocalHyperdrive method, due to lazy event querying on remote chains.

Wallet objects and Positions

Historical positions

We also expose a couple of helper functions for getting positions over time. These are exclusive to LocalHyperdrive, due to the underlying table (i.e., PositionSnapshot) only being guaranteed to keep historical data for the simulated environment.

Streamlit dashboard

codecov[bot] commented 5 months ago

Codecov Report

Attention: Patch coverage is 87.60234% with 106 lines in your changes are missing coverage. Please review.

Project coverage is 83.54%. Comparing base (9950db3) to head (dc79d09).

:exclamation: Current head dc79d09 differs from pull request most recent head fbf0dda

Please upload reports for the commit fbf0dda to get more accurate results.

Files Patch % Lines
.../agent0/core/hyperdrive/interactive/local_chain.py 50.94% 26 Missing :warning:
src/agent0/chainsync/db/hyperdrive/convert_data.py 82.87% 25 Missing :warning:
src/agent0/core/hyperdrive/interactive/chain.py 66.66% 18 Missing :warning:
src/agent0/chainsync/db/hyperdrive/interface.py 89.84% 13 Missing :warning:
...t0/core/hyperdrive/interactive/local_hyperdrive.py 78.00% 11 Missing :warning:
...c/agent0/core/hyperdrive/interactive/hyperdrive.py 94.04% 5 Missing :warning:
src/agent0/chainsync/exec/acquire_data.py 66.66% 3 Missing :warning:
src/agent0/chainsync/exec/analyze_data.py 75.00% 2 Missing :warning:
...c/agent0/chainsync/analysis/calc_position_value.py 95.65% 1 Missing :warning:
.../agent0/chainsync/dashboard/build_dashboard_dfs.py 98.38% 1 Missing :warning:
... and 1 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1467 +/- ## ========================================== + Coverage 82.28% 83.54% +1.26% ========================================== Files 90 87 -3 Lines 6019 5623 -396 ========================================== - Hits 4953 4698 -255 + Misses 1066 925 -141 ``` | [Flag](https://app.codecov.io/gh/delvtech/agent0/pull/1467/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=delvtech) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/delvtech/agent0/pull/1467/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=delvtech) | `83.54% <87.60%> (+1.26%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=delvtech#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

dpaiton commented 5 months ago

There are several related chainsync issues that may or may not be resolved by this PR; please close them if they are

Update chainsync to work with testnet https://github.com/delvtech/agent0/issues/1411

Use hyperdrivepy rust functions in chainsync https://github.com/delvtech/agent0/issues/1166

chainsync: update to use hyperdrivepy's get spot price https://github.com/delvtech/agent0/issues/927

update chainsync to have a pool column, use one db across pools https://github.com/delvtech/agent0/issues/1455

Extend chainsync to work with remote connections in interactive hyperdrive https://github.com/delvtech/agent0/issues/1456