tradingstrategy-ai / trade-executor

A Python framework for managing positions and trades in DeFi
https://tradingstrategy.ai
Other
107 stars 29 forks source link

Annual return values in `metadata` and `state` don't match #1031

Open kenkunz opened 3 months ago

kenkunz commented 3 months ago

Issue

A strategy's annual return percent value is available from both metadata and state endpoints, but the values don't match.

Background

See item "B" in this discord thread.

The annual return percent is displayed in multiple places on frontend:

Why two different data sources?

The data displayed on strategy tiles and overview page needs to be available fast and without massive data overhead, so it is included in the metadata endpoint.

The metrics table shown on the Performance page includes a lot more statistics, and is therefore included in the state endpoint (to avoid overloading metadata with too much data). The display of this data (order of elements, value formatting, labels, etc.) is fully controlled by the API – the only role that frontend plays is the table styling.

Steps to reproduce

The following steps can be reproduced for any strategy. Note that some strategies currently use live data for the metadata cagr value, while others use backtested data. When comparing the metadata and state values, be sure to compare the same source – the long_short_metrics_latest data includes both live_stats and backtested_stats. The example below uses live data for cagr, so it is compared the the live_stats data from the state endpoint.

  1. Request the cagr data from metadata endpoint:
    curl 'https://enzyme-polygon-matic-eth-usdc.tradingstrategy.ai/metadata' | jq '.summary_statistics.key_metrics.cagr'
    response
    {
     "kind": "cagr",
     "source": "live_trading",
     "value": 0.052799913985317914,
     "calculation_window_start_at": 1710974716,
     "calculation_window_end_at": 1724659698,
     "calculation_method": "historical_data",
     "unavailability_reason": null,
     "help_link": "https://tradingstrategy.ai/glossary/compound-annual-growth-rate-cagr",
     "name": null
    }
  2. Request the annualised_return_percent data from state endpoint:
    curl 'https://enzyme-polygon-matic-eth-usdc.tradingstrategy.ai/state' | jq '.stats.long_short_metrics_latest.live_stats.rows.annualised_return_percent'
    response
    {
     "kind": "annualised_return_percent",
     "source": "live_trading",
     "value": {
       "All": "5.19%",
       "Long": "5.19%",
       "Short": "-"
     },
     "calculation_window_start_at": 1710974716,
     "calculation_window_end_at": 1724659686,
     "calculation_method": null,
     "unavailability_reason": null,
     "help_link": null,
     "name": "Annualised return %"
    }
  3. Note the values from each of the endpoints.
    • expected: the values match (though the metadata value is a decimal value and the state value is a formatted percent)
    • actual: the values differ – in the above example: 0.05299… = 5.3% from metadata vs. 5.19% from state