Fix #1278: [Sim] Reframe to classify 'will price go up *anytime* in next 5 min?'

[trentmc]

Fixes #1278

[trentmc]

Some benchmarks. First two plots are fine. Something's up with the classifier performances.

Pdr profit

Tdr profit (lost this one :( )

Classifier performance

Model response

[trentmc]

Some benchmarks. Right after commit https://github.com/oceanprotocol/pdr-backend/pull/1281/commits/e475ee2ab51191d4fff5079354d02988d8a8b76d, where I cleaned up separation of HIGH and LOW classifier models. Performance plots look a bit more sane.

• Classifier accuracy is >79%. If this is true, it's amazing

Settings: my_ppss.yaml.txt

• stop loss % = take profit % = classifier threshold % = 0.2%

Plots to ignore: pdr/tdr profit vs prob_up, when prob_up < 0.5. (So far it's only plotting probs_up_UP)

2024-06-23 12:27:21,788 sim_engine INFO Iter #5000/5000 ut=1719138000000 dt=2024-06-23_10:20 ║ prob_up=0.038 pdr_profit=3.77 up + -26.43 down = -22.66 OCEAN (cumul 2407.88 OCEAN) ║ Acc=4498/5000 =  78.62% [ 77.6%,  79.7%] prcsn=0.562 recall=0.463 f1=0.503 loss=0.591 ║ tdr_profit=$  0.00 (cumul $104.35)

Pdr profit

Tdr profit

Classifier performance

Model response

[trentmc]

Some benchmarks, run right after previous benchmark.

• Identical my_psss.yaml file
• Except running on main branch (vs this PR). Main branch has stop loss / take profit but has a single model classifying up/down.

Pdr profit

Tdr profit

Model perf

Model response surface

[trentmc]

Some benchmarks. Right after commit https://github.com/oceanprotocol/pdr-backend/pull/1281/commits/a03244ed1a625c8ee40fe75abcc99f6e52cda1a6

• "Improve predictoor logic: only stake when models are confident enough"
• I've also created prob_up_MERGED, for use in predictoor bot staking, and the plots. Update plots to use that value. Therefore the plots on pdr/tdr profit vs prob_up are now sane.

Settings: same ppss as before: my_ppss.yaml.txt

• stop loss % = take profit % = classifier threshold % = 0.2%
• predictoor_ss.sim_only.others_accuracy = 0.50001

Pdr profit

Tdr profit

Classifier performance

Model response

[trentmc]

Benchmarks, with setup like above, but for more coins. Focus on trading results.

BTC (like above)

ETH

BNB

XRP

ADA

DOGE

SOL

LTC

TRX

DOT